Research map / Image generation / Fast diffusion sampling
One-Step Diffusion Distillation via Deep Equilibrium Models
Zhengyang Geng, Ashwini Pokle, J Zico Kolter
This paper tried a common change: Larger batch size
increase batch size to 2×BS · CIFAR-10 unconditional · FID: 7.42 -> 6.91 Improved
What 785 accepted papers report for this change
- 469 improved
- 107 no clear effect
- 48 mixed
- 161 hurt
- Predicting the Emergence of Induction Heads in Language Model Pretraining ICML 2026 · increase batch size (fixed context 1024) · Natural data (CC100) · induction head emergence point (updates): rho=-1.0, p<0.001Improved
- When RL Meets Adaptive Speculative Training: A Unified Training-Serving System ICML 2026 · Increase batch size from 1 to 32 · MiniMax M2.1 FP8 · speedup: 1.57x -> 1.25xHurt
- Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking ICML 2026 · increase per-device batch size to 10 using freed memory · Llama 3-8B, 128K · throughput (TPS): 2464.10 (Ulysses) -> 2526.77 (UPipe)Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill directly from noise/image pairs with GET rather than use progressive distillation | CIFAR-10 unconditional generation · CIFAR-10 unconditional | FID: 9.12 -> 6.91 | Improved |
| use GET instead of ViT-Base | CIFAR-10 unconditional generation · CIFAR-10 unconditional | FID: 11.49 -> 10.72 | Improved |
| use GET-Mini instead of ViT-Base | CIFAR-10 unconditional generation · CIFAR-10 unconditional | training speed: 4.83 iter/sec -> 5.79 iter/sec | Improved |
| use GET-Mini instead of ViT-Base | CIFAR-10 unconditional generation · CIFAR-10 unconditional | training memory per GPU: 10.1GB -> 9.2GB | Improved |
| increase GET capacity from Tiny to Base (ablation) | GET · CIFAR-10 unconditional | FID: 15.19 -> 7.42 | Improved |
| train GET-Tiny for 4× iterations (ablation) | GET-Tiny · CIFAR-10 unconditional | FID: 15.19 -> 11.47 | Improved |
| increase batch size to 2×BS (ablation) | GET-Base · CIFAR-10 unconditional | FID: 7.42 -> 6.91 | Improved |
| increase training data from 1M to 2M noise-image pairs (ablation) | class-conditional GET-Base · CIFAR-10 class-conditional | FID | Improved |
| use additive class injection instead of AdaLN-Zero (ablation) | class-conditional ViT-B · CIFAR-10 class-conditioning ablation, 200k iterations | FID: 17.19 -> 12.43 | Improved |
| add class conditioning through AdaLN-Zero (ablation) | ViT-B · CIFAR-10 class-conditioning ablation, 200k iterations | FID: 15.20 -> 17.19 | Hurt |
| use GET instead of Guided Distillation | CIFAR-10 class-conditional generation · CIFAR-10 class-conditional | FID: 7.34 -> 6.25 | Improved |
Closest papers by meaning
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Distilling ODE Solvers of Diffusion Models into Smaller Steps (CVPR 2024 · similarity 0.91)
- Data-free Distillation of Diffusion Models with Bootstrapping (ICML 2024 · similarity 0.91)
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.91)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.91)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
