Research map / Image generation / Fast diffusion sampling
SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCott, which integrates SDE solvers (ER-SDE) into consistency distillation with multi-step SDE teacher sampling, noise-strength control, and a LoRA-based conditional adversarial discriminator, enabling high-quality text-to-image generation in 2-4 steps.
Problem
Diffusion models are slow at inference; existing few-step distillation methods either produce poor samples within 4 steps or only work for one step without benefiting from extra steps. Consistency distillation has also been limited to ODE teachers that underperform SDE solvers with adequate steps.
Approach
Combine consistency distillation with SDE solvers (ER-SDE, multi-step teacher sampling, moderate noise control) so the student gets a stronger teacher, and add a LoRA-decoder U-Net discriminator conditioned on timestep and text to strengthen consistency constraints at few steps.
Key results
- 2-step SCott achieves FID 21.9 on MSCOCO-2017 5K with SD1.5, beating 1-step InstaFlow-0.9B (23.4) and 4-step UFOGen (22.1).
- On MJHQ-5K with RV5.1, 2-step SCott reaches FID 24.9 vs. 37.2 for 2-step LCM, with CLIP Score 0.301 vs. 0.296.
- 2-step SCott Coverage is 0.9114 vs. 0.7882 for LCM on MSCOCO-2017 5K, indicating higher sample diversity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase SCott inference steps from 1 to 4 | SCott · MSCOCO-2017 5K | FID: 26.8 -> 21.8 | Helped |
| Use SCott instead of LCM at 2 steps | text-to-image generation · MJHQ-5K | FID: 37.2 -> 24.9 | Helped |
| Add moderate stochastic noise with DDIM η=0.1 or η=0.2 instead of deterministic DDIM (ablation) | SCott consistency distillation · MJHQ-5K | FID: 27.4 -> 26.2; 27.4 -> 25.2 | Helped |
| Increase DDIM noise to η=0.6 (ablation) | SCott consistency distillation · MJHQ-5K | FID: 27.4 -> 29.7 | Hurt |
| Use ER-SDE 5 instead of deterministic DDIM (ablation) | SCott consistency distillation · MJHQ-5K | FID: 27.4 -> 24.9 | Helped |
| Increase training-time SDE solver steps from 1 to 3 (ablation) | SCott · MJHQ-5K | FID: 27.4 -> 24.9 | Helped |
| Add GAN loss (ablation) | SCott · MSCOCO-2017 5K | FID: 25.0 -> 21.9 | Helped |
| Replace the LoRA discriminator with a fully parameterized discriminator (ablation) | SCott · MSCOCO-2017 5K | FID: 21.9 -> 23.5 | Hurt |
| Add timestep and text conditioning to the discriminator (ablation) | SCott · MSCOCO-2017 5K | FID: 26.1 -> 21.9 | Helped |
| Use SD1.5 instead of DS-V7 as teacher (ablation) | SCott · MSCOCO-2017 5K | FID: 28.6 -> 22.1 | Helped |
Surprising findings
- Stochastic (SDE-based) consistency distillation outperforms ODE-based DDIM distillation at moderate noise levels, while excessive noise (eta=0.6) degrades results.
- Ablation shows SCott without GAN already surpasses ODE-based LCM, and the discriminator also improves LCM outputs.
Limitations
- Few-step performance at 1 step is weaker (FID 26.8), so the method relies on 2-4 steps for its strength.
- Comparisons with UFOGen, InstaFlow and ADD are limited because their training code is unavailable, and FID is noted as not always reliable for visual quality.
Experimental setup
Datasets: LAION-Aesthetics-6+ MSCOCO-2017 5K MSCOCO-2014 30K MJHQ-5K
Benchmarks: MSCOCO-2017 5K MSCOCO-2014 30K MJHQ-5K
Metrics: FID CLIP Score Coverage
Baselines: LCM InstaFlow UFOGen DMD SwiftBrush PD CAD DPM++ DDIM
Builds on: consistency distillation latent consistency models ER-SDE CTM UFOGen
Closest papers by meaning
- Diffusion Model is Effectively Its Own Teacher (CVPR 2025 · similarity 0.90)
- Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Models (CVPR 2024 · similarity 0.90)
- Consistency Diffusion Bridge Models (NeurIPS 2024 · similarity 0.90)
- Simple and Fast Distillation of Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion (ICLR 2024 · similarity 0.90)
- SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.