Research map / Image generation / Fast diffusion sampling
Improved Distribution Matching Distillation for Fast Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Improves Distribution Matching Distillation (DMD2) by removing the regression loss, stabilizing with a two time-scale update rule, adding a GAN loss on real images, and simulating multi-step inference during training, yielding strong one-step and four-step generators.
Problem
DMD requires an expensive regression loss on teacher-generated noise-image pairs for stable training, which is costly for large text-to-image models and caps student quality at the teacher's level.
Approach
Remove the regression loss and stabilize pure distribution matching by updating the fake score critic more often than the generator (two time-scale update rule); add a GAN discriminator on real vs. generated images using the fake diffusion model's features to counter teacher score errors; and train multi-step generators on student-simulated inference-time inputs (backward simulation).
Key results
- One-step ImageNet-64x64 FID of 1.28 with longer training, surpassing the EDM teacher's SDE result of 1.36.
- One-step SD v1.5 generator reaches FID 8.35 on COCO 2014 30K, improving on DMD's 11.49.
- 4-step SDXL generator reaches FID 19.32 and CLIP 0.332 on COCO 10K, rivaling the 100-step SDXL teacher (FID 19.36).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove the regression loss from DMD (ablation) | DMD on ImageNet-64×64 · ImageNet-64×64 | FID: 2.62 -> 3.48 | Hurt |
| Add TTUR (five fake-score updates per generator update) after removing regression loss (ablation) | DMD on ImageNet-64×64 · ImageNet-64×64 | FID: 3.48 -> 2.61 | Helped |
| Add the GAN loss to DMD with no regression loss and TTUR (ablation) | DMD2 on ImageNet-64×64 · ImageNet-64×64 | FID: 2.61 -> 1.51 | Helped |
| Add TTUR to GAN-only training without distribution matching (ablation) | GAN-only ImageNet training · ImageNet-64×64 | FID: 2.56 -> 2.52 | No clear effect |
| Remove the GAN loss (ablation) | DMD2 on COCO 2014 (10K prompts, SDXL; 4-step) · COCO 2014 (10K prompts, SDXL; 4-step) | FID: 19.32 -> 26.90 | Hurt |
| Remove the GAN loss (ablation) | DMD2 on COCO 2014 (10K prompts, SDXL; 4-step) · COCO 2014 (10K prompts, SDXL; 4-step) | CLIP: 0.332 -> 0.328 | Hurt |
| Remove distribution matching and use a GAN-only method (ablation) | DMD2 on COCO 2014 (10K prompts, SDXL; 4-step) · COCO 2014 (10K prompts, SDXL; 4-step) | CLIP: 0.332 -> 0.307 | Hurt |
| Remove distribution matching and use a GAN-only method (ablation) | DMD2 on COCO 2014 (10K prompts, SDXL; 4-step) · COCO 2014 (10K prompts, SDXL; 4-step) | FID: 19.32 -> 13.77 | Helped |
| Remove backward simulation (ablation) | DMD2 on COCO 2014 (10K prompts, SDXL; 4-step) · COCO 2014 (10K prompts, SDXL; 4-step) | Patch FID: 20.86 -> 24.21 | Hurt |
| Distill SD v1.5 with DMD2 instead of DMD | One-step text-to-image generation on COCO 2014 · COCO 2014 (30K prompts, SD v1.5) | FID: 11.49 -> 8.35 | Helped |
| Distill EDM with DMD2 rather than using the teacher ODE sampler | One-step image generation on ImageNet-64×64 · ImageNet-64×64 | FID: 2.32 -> 1.51 | Helped |
Surprising findings
- Naively removing the regression loss from DMD causes training instability, with fluctuating image brightness, which the two time-scale update rule fixes.
- A pure GAN loss without distribution matching yields the lowest FID on SDXL (13.77) but much worse text alignment and aesthetics.
Limitations
- Slight loss of image diversity compared to teacher models.
- Four steps still needed to match the largest SDXL teacher quality; fixed guidance scale during training.
- Training large-scale models is computationally intensive.
Experimental setup
Datasets: ImageNet-64x64 LAION-Aesthetics COCO 2014
Benchmarks: ImageNet-64x64 COCO 2014 (zero-shot) PartiPrompts
Metrics: FID CLIP score Patch FID LPIPS diversity score human preference
Baselines: DMD SDXL-Turbo SDXL-Lightning LCM-SDXL UFOGen SLAM CTM Imagine Flash SDXL teacher EDM teacher
Builds on: Distribution Matching Distillation (DMD) consistency models SDXL-Lightning Diffusion-GAN TTUR
Closest papers by meaning
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.92)
- Distribution Matching Distillation without Fake Score Network (NeurIPS 2026 · similarity 0.89)
- Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.89)
- SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation (CVPR 2024 · similarity 0.89)
- Multistep Distillation of Diffusion Models via Moment Matching (NeurIPS 2024 · similarity 0.88)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.