Research map / Image generation / Fast diffusion sampling
Importance-Aware Low-Rank Distillation of Diffusion Transformers
Denis Zavadski, Sebastian Heid, Damjan Kalšan, Stefan Roth, Carsten Rother
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use truncated SVD instead of block pruning (single-block probing) (ablation) | FLUX.dev blocks · single-block probing | mean CMMD per block: 1.18 -> 0.23 | Improved |
| use SVDtrunc instead of EcoDiff, training-free | FLUX.dev · 90% params | avg quality reduction R (%): 14.52 -> 5.24 | Improved |
| use SVDtrunc instead of EcoDiff, training-free | FLUX.dev · 80% params | avg quality reduction R (%): 45.99 -> 6.69 | Improved |
| use SVDtrunc instead of EcoDiff, training-free | FLUX.dev · 70% params | avg quality reduction R (%): 85.50 -> 9.86 | Improved |
| use SVDtrunc instead of block pruning, training-free | FLUX.dev · 90% params | avg quality reduction R (%): 1.00 -> 5.24 | Hurt |
| use SVDtrunc instead of block pruning, training-free | FLUX.dev · 70% params | avg quality reduction R (%): 94.52 -> 9.86 | Improved |
| add importance-aware allocation vs random (ablation) | SVDtrunc · 57% params | avg quality reduction R (%): 5.68 -> 2.60 | Improved |
| add importance-aware allocation vs uniform (ablation) | SVDtrunc · 57% params | avg quality reduction R (%): 3.19 -> 2.60 | Improved |
| add importance-aware allocation vs uniform (ablation) | SVDtrunc · 49% params | avg quality reduction R (%): 6.23 -> 4.68 | Improved |
| add importance-aware allocation vs uniform (ablation) | SVDtrunc · 68% params | avg quality reduction R (%): 1.46 -> 0.75 | Improved |
| add modular knowledge distillation to rectified-flow fine-tuning (ablation) | SVDtrunc · 68% params | avg quality reduction R (%): 2.12 -> 0.93 | Improved |
| use MKD only vs combined objective (ablation) | SVDtrunc · 68% params | avg quality reduction R (%): 0.64 -> 0.93 | No clear effect |
Closest papers by meaning
- Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers (CVPR 2025 · similarity 0.92)
- Scaling Diffusion Transformers Efficiently via $\mu$P (NeurIPS 2025 · similarity 0.91)
- SparseDiT: Token Sparsification for Efficient Diffusion Transformer (NeurIPS 2025 · similarity 0.90)
- EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing (ICLR 2025 · similarity 0.89)
- SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer (ICML 2025 · similarity 0.89)
- Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
