Research map / Image generation / Fast diffusion sampling
Simple and Fast Distillation of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Simple and Fast Distillation (SFD) of diffusion models that distills over the whole teacher trajectory with a few tuned factors (t_min, AFS, L1 loss, DPM-Solver++(3M) teacher), cutting fine-tuning cost up to 1000x, plus SFD-v with a step-condition for variable NFE.
Problem
Distillation-based diffusion samplers need very expensive fine-tuning (often 100+ A100 hours) to reach good quality at a fixed small NFE, and existing methods fine-tune at many timestamps that are never used in sampling.
Approach
Fine-tune only the timestamps used in sampling, distill the whole teacher trajectory globally so the student learns to correct its own accumulated errors, and tune small factors (DPM-Solver++(3M) teacher, slightly larger t_min, analytical first step, L1 loss); a step-condition lets one model sample at variable NFE.
Key results
- SFD achieves FID 4.53 at NFE=2 on CIFAR-10 with 0.64 A100 hours of fine-tuning, versus about 1156 hours for consistency distillation.
- SFD matches progressive distillation quality (FID 4.51 at NFE=2) at roughly 100-200x lower fine-tuning cost on CIFAR-10.
- On ImageNet 64x64, SFD-v reaches FID 9.47 at NFE=2 and 4.21 at NFE=5 with a single distilled model supporting variable NFE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace local trajectory distillation with global SFD (ablation) | Trajectory distillation · CIFAR10, 2-NFE distillation | FID: 46.84 -> 20.88 | Helped |
| Use DPM++(3M) as the teacher solver instead of Heun (ablation) | SFD · CIFAR10, 2-NFE distillation | FID: 20.88 -> 11.65 | Helped |
| Increase t_min from 0.002 to 0.006 (ablation) | SFD with DPM++(3M), no AFS · CIFAR10, 2-NFE distillation | FID: 11.65 -> 10.93 | Helped |
| Add AFS (ablation) | SFD with DPM++(3M), t_min=0.002, L2 loss · CIFAR10, 2-NFE distillation | FID: 11.65 -> 7.17 | Helped |
| Use L1 loss instead of L2 loss (ablation) | SFD with DPM++(3M), t_min=0.006, AFS · CIFAR10, 2-NFE distillation | FID: 5.67 -> 4.57 | Helped |
| Use LPIPS loss instead of L2 loss (ablation) | SFD with DPM++(3M), t_min=0.006, AFS · CIFAR10, 2-NFE distillation | FID: 5.67 -> 5.10 | Hurt |
| Use Pseudo-Huber loss instead of L2 loss (ablation) | SFD with DPM++(3M), t_min=0.006, AFS · CIFAR10, 2-NFE distillation | FID: 5.67 -> 4.90 | Helped |
| Use step-condition instead of t_next-condition (ablation) | Variable-NFE distillation · Variable-NFE image generation | FID | Helped |
| Add second-stage one-NFE distillation (ablation) | SFD · CIFAR10, NFE=1 | FID | Helped |
| Set teacher sampling steps K to 4 (ablation) | SFD · CIFAR10, NFE=2 | FID: 6.51 -> 4.57 | Helped |
| Train SFD-v with variable sampling steps (ablation) | SFD · CIFAR10, NFE=2 | FID: 4.53 -> 4.28 | Helped |
| Distill Stable Diffusion at guidance scale 1, then sample at guidance scale 7.5 | SFD-v · Stable Diffusion v1.5, MS-COCO val | FID-5K | Helped |
Surprising findings
- Fine-tuning at one timestamp improves the gradient field at other, distant timestamps, so fine-tuning on a fine-grained schedule is unnecessary.
- One more inaccurate step (analytical first step) outperforms one fewer step: AFS with N=3 beats N=4 without AFS at 2 NFE.
Limitations
- FID results do not match state-of-the-art methods, as acknowledged by the authors.
- Training costs for baselines are estimated by re-implementation rather than measured from original runs.
- Second-stage one-NFE distillation is needed for competitive 1-NFE results and the time schedule is not tailored to the method.
Experimental setup
Datasets: CIFAR-10 ImageNet 64x64 LSUN-Bedroom 256x256 MS-COCO validation
Benchmarks: CIFAR-10 ImageNet 64x64 LSUN-Bedroom 256x256 MS-COCO validation
Metrics: FID CLIP score training time (A100 hours)
Baselines: Progressive distillation Guided progressive distillation Consistency distillation Consistency trajectory models DDIM DPM-Solver++ AMED-Plugin GITS SnapFusion DPM-Solver++(2M)
Builds on: progressive distillation consistency models DPM-Solver++ EDM Latent Diffusion / Stable Diffusion
Closest papers by meaning
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.92)
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.92)
- Distilling ODE Solvers of Diffusion Models into Smaller Steps (CVPR 2024 · similarity 0.91)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.91)
- Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation (ICML 2024 · similarity 0.91)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.