Research map / Image generation / Fast diffusion sampling
One-Step Diffusion Distillation through Score Implicit Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills pre-trained diffusion models (EDM on CIFAR-10, PixArt-α 0.6B DiT for text-to-image) into one-step generators data-free via a score-gradient theorem that makes gradients of score-based divergences tractable.
Problem
Distilling multi-step diffusion models into one-step generators while preserving sample quality, without needing real training data, and with a tractable training objective.
Approach
Shows that gradients of a broad class of score-based divergences between a generator and a teacher diffusion model can be computed exactly via a score-gradient theorem (score-projection identity with stop-gradient), yielding a tractable Score Implicit Matching loss; uses a Pseudo-Huber distance whose gradient adaptively normalizes the loss, and recovers SiD for the squared L2 distance.
Key results
- On CIFAR-10, SIM achieves FID 2.06 unconditional (1 step) and 1.96 class-conditional (1 step).
- SIM-DiT-600M reaches aesthetic score 6.42 in one step, matching the 30-step PixArt-α teacher's 5.97 and beating SDXL-Turbo (5.33), SDXL-Lightning (5.34) and Hyper-SDXL (5.85).
- Distilling PixArt-α took about 2 days on 4 A100-40G GPUs, data-free for the generator training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Pseudo-Huber distance instead of squared L2 distance (ablation) | SIM on CIFAR-10 · CIFAR-10 unconditional | FID | Helped |
| Use the modified log-normal time distribution instead of Karr time distribution (ablation) | SIM generator training · CIFAR-10 | FID | Helped |
| Use constant generator loss weighting instead of SiD weighting (ablation) | SIM generator training · CIFAR-10 | FID | Helped |
| Train with a learning rate of 1e-4 rather than the smaller-rate setting (ablation) | SIM, SiD, and DI on EDM · CIFAR-10 unconditional | FID stability | Mixed |
| Use SIM instead of SiD at matched training iterations | One-step EDM distillation · CIFAR-10 unconditional | FID | Helped |
| Concentrate the noise distribution around smaller values (ablation) | SIM-DiT-600M · Text-to-image generation | image brightness | Hurt |
| Use a slightly larger noise distribution (ablation) | SIM-DiT-600M · Text-to-image generation | structural stability | Hurt |
| Increase generator initialization noise level t* (ablation) | SIM-DiT-600M · Text-to-image generation | convergence speed | Helped |
| Distill into one step instead of using the 30-step PixelArt-α teacher | SIM-DiT-600M · COCO-2017 validation | aesthetic score: 5.97 -> 6.42 | Helped |
| Distill into one step instead of using the 30-step PixelArt-α teacher | SIM-DiT-600M* · SAM-LLaVA-Caption10M | aesthetic score: 5.93 -> 5.91 | No clear effect |
| Distill into one step instead of using the 30-step PixelArt-α teacher | SIM-DiT-600M* · SAM-LLaVA-Caption10M | Image Reward: 0.53 -> 0.44 | Hurt |
Surprising findings
- Pseudo-Huber SIM stays stable at large learning rates where SiD and DI are unstable.
- The SiD-style Karras time schedule did not work well in practice; a modified log-normal time distribution performed better.
Limitations
- Fails to generate tiny human faces, correct arms and fingers, and sometimes the wrong number of objects.
- Image Reward of SIM-DiT-600M is slightly lower than the teacher's.
- Only evaluated on diffusion (EDM, PixArt-α) teachers; not extended to flow-matching models.
Experimental setup
Datasets: CIFAR-10 SAM-LLaVA-Caption10M
Benchmarks: CIFAR-10 COCO-2017 validation SAM-LLaVA-Caption10M
Metrics: FID Inception Score Aesthetic Score PickScore ImageReward User preference win rate
Baselines: DDPM Diff-Instruct SiD CTM DMD SDXL-Turbo SDXL-Lightning Hyper-SDXL LCM TCD PeRFlow Hyper-SD
Builds on: Diff-Instruct Score identity Distillation (SiD) EDM PixArt-α
Closest papers by meaning
- Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation (ICML 2024 · similarity 0.92)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.92)
- Evaluating the design space of diffusion-based generative models (NeurIPS 2024 · similarity 0.91)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.91)
- Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation (ICLR 2025 · similarity 0.91)
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.