Research map / Image generation / Fast diffusion sampling
Data-free Distillation of Diffusion Models with Bootstrapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BOOT, a data-free distillation method that trains a time-conditioned single-step student of a diffusion model from pure Gaussian noise using bootstrapped Signal-ODE objectives, applied to image benchmarks and text-to-image models.
Problem
Distilling diffusion models into fast single-step generators usually needs either synthetic teacher-generated datasets or real training data, which is costly or unavailable (e.g., for large text-to-image models).
Approach
A student gθ(ϵ, t) maps fixed Gaussian noise to the teacher's intermediate signal state at any time t, trained via finite-difference bootstrapping from a derived Signal-ODE between adjacent timesteps, plus a boundary loss at t_max; no real or synthetic data is needed and no EMA is used.
Key results
- On CIFAR-10 one-step BOOT with LPIPS loss achieves FID 4.38 vs 3.55 for CM (800K iterations) and 8.34 for progressive distillation, the first data-free single-step result.
- On FFHQ 64x64 and ImageNet 64x64, one-step BOOT gets FID 9.0 and 12.3, versus 7.6 and 13.7 for 50-step DDIM, at 54 and 34 fps.
- With the SD teacher, BOOT reaches CLIP score 0.254 on COCO2017 vs 0.262 for 50-step DDIM, and FID 30.21 vs 111.55 for LCM on DiffusionDB prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill to a single step with BOOT instead of using 50-step DDIM | Diffusion model on FFHQ 64 × 64 · FFHQ 64 × 64 | FID: 7.6 -> 9.0 | Hurt |
| Distill to a single step with BOOT instead of using 10-step DDIM | Diffusion model on FFHQ 64 × 64 · FFHQ 64 × 64 | FID: 18.3 -> 9.0 | Helped |
| Distill to a single step with BOOT instead of using 50-step DDIM | Class-conditional diffusion model · ImageNet 64 × 64 | FID: 13.7 -> 12.3 | Helped |
| Distill to a single step with BOOT instead of using 10-step DDIM | Class-conditional diffusion model · ImageNet 64 × 64 | FID: 18.3 -> 12.3 | Helped |
| Use LPIPS loss instead of L2 loss (ablation) | BOOT · CIFAR-10 | FID: 6.88 -> 4.38 | Helped |
| Use BOOT instead of CM | Single-step diffusion distillation · CIFAR-10 | FID: 3.55 -> 4.38 | Hurt |
| Use BOOT instead of PD | Single-step diffusion distillation · CIFAR-10 | FID: 8.34 -> 4.38 | Helped |
| Distill StableDiffusion to one step with BOOT instead of using 50-step DDIM | StableDiffusion v2.1-Base · COCO2017 validation | CLIP score: 0.262 -> 0.254 | Hurt |
| Distill StableDiffusion with BOOT instead of LCM | StableDiffusion · DiffusionDB | FID: 111.55 -> 30.21 | Helped |
| Add the boundary loss (ablation) | BOOT · Text-to-image generation | generation diversity | Helped |
| Use uniform time sampling instead of progressive time training (ablation) | BOOT · Text-to-image generation | generation artifacts | Helped |
Surprising findings
- Progressive time training, which intuitively should be more efficient, introduces more artifacts than uniform time sampling.
- Without a boundary loss, the student shows severe mode collapse and over-saturation, indicating the boundary condition is critical for text-to-image.
Limitations
- Sample quality is lower than distillation methods that use ground-truth or synthetic data (e.g., CM, TRACT).
- Only supports single-step students; multi-step generation is not supported in the current design.
- Training is slower due to CFG and boundary loss requiring extra teacher NFEs.
Experimental setup
Datasets: DiffusionDB CIFAR-10 FFHQ ImageNet COCO2017
Benchmarks: CIFAR-10 FFHQ 64x64 ImageNet 64x64 COCO2017 DiffusionDB
Metrics: FID precision recall CLIP score fps
Baselines: DDIM DDPM DPM-solver-2 DEIS Direct distillation DFNO ReFlow Progressive distillation Consistency Model LCM
Builds on: consistency models progressive distillation DDIM EDM classifier-free guidance
Closest papers by meaning
- DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture (CVPR 2025 · similarity 0.93)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.91)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.91)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.90)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.