Research map / Image generation / Autoregressive and latent generation
Representation Alignment for Diffusion Transformers without External Components
Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, Jingdong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SRA self-alignment (3->8) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 33.02 -> 29.10 | Improved |
| align layer 3 to deeper teacher layer 8 instead of the same layer 3->3 (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 37.08 -> 29.10 | Improved |
| align student layer 6 with teacher layer 10 instead of early layer 3->8 (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 29.10 -> 34.85 | Hurt |
| add projection head (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 34.23 -> 29.10 | Improved |
| use teacher with lower noise (interval 0.1) instead of equal noise (0.0) (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 31.07 -> 29.55 | Improved |
| use fixed interval 0.2 instead of 0.1 (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 29.55 -> 30.70 | Hurt |
| vary lambda from 0.1 to 0.4 (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 30.65, 29.10, 29.28, 29.75 | No clear effect |
| use EMA alpha 0.9999 instead of student copy alpha 0.0 (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 35.71 -> 29.10 | Improved |
| use EMA alpha 0.9999 instead of 0.996 -> 1.0 schedule (ablation) | SRA on SiT-B/2 · ImageNet 256 no CFG, 400K | FID: 33.17 -> 29.10 | Improved |
| replace SiT-XL/2 baseline with SiT + SRA at 800 epochs | SRA on SiT-B/2 · ImageNet 256 with CFG | FID: 2.06 -> 1.58 | Improved |
| use SRA instead of REPA at 800 epochs | SRA on SiT-B/2 · ImageNet 256 with CFG | FID: 1.42 -> 1.58 | Hurt |
| apply SRA to MMDiT | SRA on SiT-B/2 · COCO2014 | FID: 5.86 -> 4.85 | Improved |
Closest papers by meaning
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.93)
- AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers (NeurIPS 2026 · similarity 0.92)
- REPA Works Until It Doesn’t: Early-Stopped, Holistic Alignment Supercharges Diffusion Training (NeurIPS 2025 · similarity 0.90)
- SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformer (CVPR 2024 · similarity 0.90)
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.89)
- TinyFusion: Diffusion Transformers Learned Shallow (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
