Research map / Image generation / Fast diffusion sampling
Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Flash Diffusion, a LoRA-based distillation method combining a distillation loss, latent adversarial loss, and distribution matching loss to make pre-trained conditional diffusion models generate images in 2-4 steps across tasks and backbones.
Problem
Pre-trained diffusion models need many sampling steps; existing distillation methods are costly, tailored to text-to-image, and unclear across tasks, conditionings, and architectures.
Approach
Train a LoRA student initialized from the teacher to predict a multi-step CFG teacher denoised latent in one step, with a latent (encoder-only) discriminator on re-noised samples and a DMD loss where the student itself serves as the fake score model, sampling timesteps from a mixture of Gaussians over K uniform timesteps emphasizing few-step timesteps.
Key results
- On COCO2014 with SD1.5 teacher, Flash Diffusion reaches FID 12.27 at 2 NFEs, reported as SOTA for few-step generation.
- On COCO2017 with SD1.5 teacher, FID 22.6 at 2 NFEs with CLIP score 30.6 (2 NFEs) and 31.1 (4 NFEs).
- Trains only 26.4M LoRA parameters for about 26 H100 GPU hours on SD1.5; Flash SDXL reaches FID 21.6 with 4 NFEs versus 24.6 for SDXL-Lightning and 27.8 for Hyper-SD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the adversarial loss (ablation) | Flash Diffusion · COCO2017, 2 NFEs | FID | Helped |
| Add the distribution matching loss (ablation) | Flash Diffusion · COCO2017, 2 NFEs | CLIP score | Helped |
| Remove the distillation loss (ablation) | Flash Diffusion · COCO2017, 2 NFEs | training stability | Hurt |
| Use MSE instead of LPIPS for the distillation loss (ablation) | Flash Diffusion · COCO2017, 2 NFEs | FID | Helped |
| Use MSE instead of LPIPS for the distillation loss (ablation) | Flash Diffusion · COCO2017, 2 NFEs | CLIP score | Helped |
| Use LSGAN instead of Hinge or WGAN (ablation) | Flash Diffusion · COCO2017, 2 NFEs | training stability | Helped |
| Use the proposed timestep distribution instead of uniform or Gaussian sampling (ablation) | Flash Diffusion · COCO2017, 2 NFEs | FID | Helped |
| Sample more than 4 distinct timesteps instead of using the sharp 4-timestep distribution (ablation) | Flash Diffusion · COCO2017, 2 NFEs | FID | Helped |
| Use a fixed teacher guidance scale instead of uniformly sampling it from a range (ablation) | Flash Diffusion · COCO2017, 2 NFEs | FID | Mixed |
Surprising findings
- Training with only the adversarial and DMD losses diverged, showing the distillation loss is essential for stability.
- Restricting timestep sampling to only the 4 inference timesteps reduced diversity and hurt FID, so sampling a wider range of timesteps helped.
Experimental setup
Datasets: LAION (aesthetic score > 6, re-captioned with CogVLM)
Benchmarks: COCO2014 COCO2017
Metrics: FID CLIP score
Baselines: LCM SDXL-Lightning Hyper-SD SDXL-Turbo Pixart-LCM SD3 teacher
Builds on: latent consistency models distribution matching distillation progressive distillation classifier-free guidance
Closest papers by meaning
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.91)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.90)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.90)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.89)
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.