Research map / Image generation / Autoregressive and latent generation
REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, Liang Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| end-to-end tune VAE with REPA loss | REPA with SiT-XL · ImageNet 256 (80 epochs, no CFG) | gFID: 7.90 -> 4.07 | Improved |
| end-to-end tune VAE with REPA loss | REPA with SiT-XL · ImageNet 256 (20 epochs, no CFG) | gFID: 19.40 -> 12.83 | Improved |
| backpropagate diffusion loss to VAE (naive E2E) instead of using stop-gradient (ablation) | REPA-E · ImageNet 256 (SiT-L, 100K) | gFID: 16.3 -> 444.1 | Hurt |
| add batch-norm latent normalization (ablation) | REPA-E · ImageNet 256 (SiT-L, 100K) | gFID: 18.1 -> 16.3 | Improved |
| add GAN regularization loss on VAE (ablation) | REPA-E · ImageNet 256 (SiT-L, 100K) | gFID: 19.2 -> 16.3 | Improved |
| add REPA-E to SiT-B | REPA · ImageNet 256 (SiT-B, no CFG) | gFID: 49.5 -> 34.8 | Improved |
| add REPA-E to SiT-XL | REPA · ImageNet 256 (SiT-XL, no CFG) | gFID: 19.4 -> 12.8 | Improved |
| use CLIP-L as REPA target with REPA-E | REPA · ImageNet 256 (SiT-L, CLIP-L encoder) | gFID: 29.2 -> 23.4 | Improved |
| add REPA-E to VA-VAE | REPA · ImageNet 256 (SiT-L, VA-VAE) | gFID: 12.8 -> 11.1 | Improved |
| add REPA-E to IN-VAE | REPA · ImageNet 256 (SiT-L, IN-VAE) | gFID: 22.7 -> 12.7 | Improved |
| apply REPA alignment at 6th layer with REPA-E | REPA · ImageNet 256 (SiT-L, alignment depth 6) | gFID: 23.0 -> 16.4 | Improved |
| initialize VAE from pretrained SD-VAE instead of from scratch (ablation) | REPA-E · ImageNet 256 (400K iters, no CFG) | gFID: 4.34 -> 4.07 | Improved |
Later papers that compared with this method
10 later accepted papers report REPA-E as a baseline in their results tables; 6 of them report a better number on most of the results they share with it.
- STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation (NeurIPS 2026 · better on 5 of 5 shared results)
- Improved Baselines with Representation Autoencoders (NeurIPS 2026 · better on 3 of 3 shared results)
- Guiding a Diffusion Transformer with the Internal Dynamics of Itself (CVPR 2026 · better on 1 of 3 shared results)
- Towards Scalable Pre-training of Visual Tokenizers for Generation (ECCV 2026 · better on 2 of 2 shared results)
- Stable Velocity: A Variance Perspective on Flow Matching (ICML 2026 · better on 0 of 1 shared result)
- What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion (NeurIPS 2026 · better on 1 of 1 shared result)
- Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusion (CVPR 2026 · better on 1 of 1 shared result)
- SpeeDiff: Scalable Pixel-Anchored End-to-End Latent Diffusion Model (CVPR 2026 · better on 1 of 1 shared result)
2 more not listed.
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.89)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.88)
- REPA Works Until It Doesn’t: Early-Stopped, Holistic Alignment Supercharges Diffusion Training (NeurIPS 2025 · similarity 0.88)
- Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis (ICLR 2025 · similarity 0.87)
- AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers (NeurIPS 2026 · similarity 0.87)
- DiffEnc: Variational Diffusion with a Learned Encoder (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
