Research map / Image generation / Fast diffusion sampling
Simple Distillation for One-Step Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EM Distillation (EMD), a maximum-likelihood EM-based method that distills a pre-trained diffusion teacher into a one-step generator using reparametrized joint Langevin MCMC over (epsilon, z) with noise cancellation.
Problem
Distilling slow multistep diffusion models into one-step generators while avoiding mode collapse (mode-seeking divergences) and performance degradation at very few steps.
Approach
Derive distillation as maximum likelihood via EM: the E-step samples jointly over (data, latent) pairs initialized from student samples using short-run Langevin MCMC in a reparametrized (epsilon, z) space, and the M-step regresses the generator onto noise-canceled MCMC samples; this interpolates between mode-seeking (VSD/Diff-Instruct, with 1 step) and mode-covering objectives as the number of MCM
Key results
- EMD-16 reaches FID 2.20 on ImageNet 64x64 class-conditional one-step generation, beating DMD (2.62) and the Diff-Instruct-equivalent EMD-1 baseline (3.1).
- EMD-16 reaches FID 6.0 on ImageNet 128x128 one-step generation, versus 7.0 for CD and 6.3 for EMD-1.
- EMD-8 achieves FID 9.66 on zero-shot MSCOCO one-step text-to-image generation distilled from SD v1.5, versus 11.49 for DMD (tCFG=3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase joint Langevin sampling from 1 to 16 steps (ablation) | EMD · ImageNet 64×64 class-conditional | FID: 3.1 -> 2.20 | Helped |
| increase joint Langevin sampling from 1 to 16 steps (ablation) | EMD · ImageNet 64×64 class-conditional | Recall: 0.55 -> 0.59 | Helped |
| increase joint Langevin sampling from 1 to 16 steps (ablation) | EMD · ImageNet 128×128 class-conditional | FID: 6.3 -> 6.0 | Helped |
| increase joint Langevin sampling from 1 to 16 steps (ablation) | EMD · ImageNet 128×128 class-conditional | Inception Score: 134 ± 2.75 -> 140 ± 2.83 | Helped |
| use EMD-16 instead of the strongest listed one-step baseline | One-step image generation · ImageNet 64×64 class-conditional | FID: 2.62 -> 2.20 | Helped |
| use EMD-8 with tCFG=2 instead of EMD-1 with tCFG=2 (ablation) | EMD text-to-image · MSCOCO FID-30k | FID: 9.78 -> 9.66 | Helped |
| use EMD-8 (tCFG=2) instead of DMD (tCFG=3) | One-step text-to-image generation · MSCOCO FID-30k | FID: 11.49 -> 9.66 | Helped |
| increase the number of Langevin steps per distillation iteration (ablation) | EMD · ImageNet 64×64 | FID | Helped |
| increase the number of Langevin steps per distillation iteration (ablation) | EMD · ImageNet 64×64 | Recall | Helped |
| sample jointly in reparameterized (ε,z) space instead of sampling only ε (ablation) | EMD-8 · ImageNet 64×64, 100k training steps | FID: 2.829 -> 2.77 | Helped |
| sample jointly in reparameterized (ε,z) space instead of sampling only ε (ablation) | EMD-8 · ImageNet 64×64, 100k training steps | Inception Score: 62.31 -> 62.98 | Helped |
| use joint (x,z) sampling instead of reparameterized (ε,z) sampling (ablation) | EMD-8 · ImageNet 64×64, 100k training steps | FID: 2.77 -> 3.11 | Hurt |
Surprising findings
- Accumulated Langevin noise removal (noise cancellation) substantially reduces gradient variance and speeds convergence, and the paper finds it critical to success.
- Sampling only epsilon with z fixed matches joint sampling at few steps but falls behind with more steps, while the (x,z)-corrector struggles to match the baseline.
Limitations
- Requires student initialized from the teacher; training from random initialization gave no competitive results.
- Sensitive to the choice of initial timestep t* (or lambda*).
- Additional training cost from multi-step MCMC per iteration; MCMC step size needs tuning.
Experimental setup
Datasets: ImageNet 64x64 ImageNet 128x128 LAION-Aesthetics-6.25+ MSCOCO
Benchmarks: ImageNet 64x64 ImageNet 128x128 MSCOCO-30k
Metrics: FID Inception Score Recall CLIP score
Baselines: Diff-Instruct DMD CD Multistep CD BOOT TRACT UFOGen LCM-LoRA InstaFlow DPM Solver
Builds on: Diff-Instruct Variational Score Distillation EDM Stable Diffusion v1.5
Closest papers by meaning
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.93)
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.92)
- EM Distillation for One-step Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.90)
- Reducing Spatial Fitting Error in Distillation of Denoising Diffusion Models (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.