Research map / Image generation / Fast diffusion sampling
Distillation for few-step generation
Distill diffusion models and datasets into fast generators using distribution matching.
84 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Fast diffusion sampling, keyword cluster “steps · distillation · time”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
16 papers in 2023–24 and 63 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.5% to 4.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Professor Forcing: A New Algorithm for Training Recurrent Networks (NeurIPS 2016 · 321 citations)
- 2018 · Parallel WaveNet: Fast High-Fidelity Speech Synthesis (ICML 2018 · 335 citations)
- 2019 · HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models (NeurIPS 2019 · 98 citations)
- 2021 · Show Me How To Revise: Improving Lexically Constrained Sentence Generation with XLNet (AAAI 2021 · 16 citations)
- 2023 · Real-Time Controllable Denoising for Image and Video (CVPR 2023 · 28 citations)
- 2024 · Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · 43 citations)
- 2025 · Let's Verify and Reinforce Image Generation Step by Step (CVPR 2025 · 4 citations)
- 2026 · SAR-DisentDM: A Semantic-Disentangled Diffusion Model for Limited-Data SAR Image Synthesis (AAAI 2026 · 3 citations)
Earliest papers here
- Professor Forcing: A New Algorithm for Training Recurrent Networks (NeurIPS 2016 · 321 citations)
- Parallel WaveNet: Fast High-Fidelity Speech Synthesis (ICML 2018 · 335 citations)
- HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models (NeurIPS 2019 · 98 citations)
What has been tried
45 papers here report 522 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 6%, hurt 20%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Remove the regression loss from DMD | ImageNet-64×64 · FID | 2.62 -> 3.48 | Hurt | Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · 43 citations) |
| Add minimax representativeness constraint instead of aligning embeddings to the distribution center | ImageWoof, 10 IPC · top-1 accuracy | 34.4±1.1 -> 37.4±0.4 | Helped | Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · 27 citations) |
| Remove EMA from the teacher network | CIFAR-10 · FID | Helped | Improved Techniques for Training Consistency Models (ICLR 2024 · 13 citations) | |
| Add adversarial consistency loss | Stable Diffusion XL, 1 step · HPS | 0.251 -> 0.280 | Helped | Phased Consistency Models (NeurIPS 2024 · 11 citations) |
| Use 10 perturbation directions instead of 5 or 20 | FFHQ-256, 50K iterations · FIDearly | 12.46 ± 0.2; 12.47 ± 0.2 -> 12.08 ± 0.3 | Helped | Diversity-aware Channel Pruning for StyleGAN Compression (CVPR 2024 · 10 citations) |
| Use a window size of 7 rather than smaller or larger windows | DreamFusion gallery (30 prompts) · Speedup | Mixed | DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling (CVPR 2024 · 9 citations) | |
| Fine-tune the ORM using ranking data rather than use it zero-shot | GenEval · Overall | 0.61 -> 0.63 | Helped | Let's Verify and Reinforce Image Generation Step by Step (CVPR 2025 · 4 citations) |
| Add CAN to UViT-S/2 | ImageNet 256×256 · FID | 28.32 -> 8.82 | Helped | Condition-Aware Neural Network for Controlled Image Generation (CVPR 2024 · 4 citations) |
| Add multi-scale dual-objective GAN training | Image generation · fine-grained detail and image coherence | Helped | NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training (CVPR 2025 · 3 citations) | |
| Add the temporal feature accumulator | Six-scene average · PSNR | 29.55 -> 32.13 | Helped | High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction (AAAI 2024 · 3 citations) |
| Add the center-copy operation | WikiArt, 11.7x · FID | 33.735 -> 36.326 | Hurt | Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach (ICLR 2024 · 3 citations) |
| add the auxiliary unlabeled dataset to student training | Bistro, Zero-Day, Classroom, Dining room · PSNR | 24.58 -> 25.36; 25.97 -> 26.95; 26.54 -> 30.62; 32.43 -> 35.02 | Helped | Real-Time Neural Denoising with Render-Aware Knowledge Distillation (AAAI 2025 · 2 citations) |
What reviewers will expect
Datasets and baselines named in the 34 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 7 of 34 papers
- ImageNet 64x64 · 5 of 34 papers
- ImageNet 512x512 · 3 of 34 papers
- ImageNet-1k · 2 of 34 papers
- ImageNet · 2 of 34 papers
Baselines
- iCT · 5 of 34 papers
- iCT-deep · 4 of 34 papers
- Diff-Instruct · 4 of 34 papers
- StyleGAN-XL · 4 of 34 papers
- DMD2 · 3 of 34 papers
- CTM · 3 of 34 papers
- DiT · 2 of 34 papers
- ECT · 2 of 34 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 79 papers; from OpenAlex and the papers' affiliations).
- National University of Singapore · 5 papers
- Shanghai Jiao Tong University · 5 papers
- Massachusetts Institute of Technology · 4 papers
- University of Texas at Austin · 4 papers
- The Chinese University of Hong Kong · 4 papers
- Hong Kong University of Science and Technology · 4 papers
- Shanghai AI Laboratory · 3 papers
- Shanghai AI Lab · 3 papers
- University of Science and Technology of China · 3 papers
- University of California, Los Angeles · 2 papers
Use it
Download BibTeX of 28 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Training-free diffusion acceleration (257 papers)
- Fast diffusion ODE solvers (120 papers)
- Accelerated diffusion policies (23 papers)
Elsewhere in the topic
- Diffusion theory and design space (252 papers)
- Diffusion models as perception tools (198 papers)
- Masked diffusion language model decoding (147 papers)
- Guidance for conditional diffusion (123 papers)
- Diffusion theory and learned samplers (121 papers)
- Distributional control of diffusion (109 papers)
- Denoising diffusion probabilistic models (97 papers)
- Flow matching distillation (95 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).