Research map / Image generation / Autoregressive and latent generation
EDiT: Efficient Diffusion Transformers with Linear Compressed Attention
Philipp Becker, Abhinav Mehrotra, Ruchika Chavhan, Malcolm Chadwick, Luca Morreale, Mehdi Noroozi, Alberto Gil C. P. Ramos, Sourav Bhattacharya
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ConvFusion on Q and K without V compression (vs full EDiT) (ablation) | EDiT · 1024x1024 | FID (Inception-v3): 7.76 vs 7.82 | No clear effect |
| add Spatial Compressor on K and V together with ConvFusion on Q (ablation) | EDiT · 1024x1024 | FID (Inception-v3): 26.59 -> 7.82 | Improved |
| add Spatial Compressor on K and V together with ConvFusion on Q (ablation) | EDiT · 512x512 | FID (Inception-v3): 14.53 -> 7.06 | Improved |
| compress keys and values with Spatial Compressor (ablation) | EDiT · 1024x1024 | FID (Inception-v3): 7.70 vs 7.82 | No clear effect |
| compress keys and values (latency) (ablation) | EDiT · 1024 image on RTX 3090 | latency (s): 4.3 -> 4.15 | Improved |
| add hybrid attention with ConvFusion and Spatial Compressor instead of basic linear functions (ablation) | MM-EDiT · 512x512 | FID (Inception-v3): 20.98 -> 11.60 | Improved |
| use hybrid attention instead of fully linear attention (SANA-MM-DiT) | MM-EDiT · 512x512 | FID (Inception-v3): 14.94 -> 11.60 | Improved |
| use heuristic adjustment instead of theoretically correct eta^Lin (ablation) | MM-EDiT · 512x512 | FID (Inception-v3): 13.05 -> 11.60 | Improved |
| replace PixArt-Sigma with EDiT linear compressed attention | EDiT · 1024x1024 | FID (Inception-v3): 7.09 -> 7.82 | Hurt |
| replace PixArt-Sigma with EDiT linear compressed attention | EDiT · 4096x4096 | latency (s): 4.770 -> 0.461 | Improved |
| replace PixArt-Sigma with EDiT linear compressed attention | EDiT · 8192x8192 | latency (s): 72.96 -> 1.693 | Improved |
| replace SD-v3.5M with MM-EDiT | MM-EDiT · 4096x4096 | latency (s): 5.603 -> 0.623 | Improved |
Closest papers by meaning
- CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up (NeurIPS 2025 · similarity 0.90)
- DiTFastAttn: Attention Compression for Diffusion Transformer Models (NeurIPS 2024 · similarity 0.90)
- Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks (AAAI 2024 · similarity 0.89)
- Exploring Diffusion Transformer Designs via Grafting (NeurIPS 2025 · similarity 0.88)
- Rare Text Semantics Were Always There in Your Diffusion Transformer (NeurIPS 2025 · similarity 0.88)
- DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
