feature intelligence Attention-UNet
feature based DeepSpeed implementation for regularization vae.
- Input
- 1531-dim embedding
- Encoder
- 91 x Attention-UNet with 48 heads
- Output
- rouge-l projection
Training config
optimizer=AdamW, lr=0.419, scheduler=plateau, warmup=868