Research map / Image generation / Fast diffusion sampling
Fast ODE-based Sampling for Diffusion Models in Around 5 Steps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AMED-Solver, a single-step ODE solver for diffusion sampling that learns a mean direction (intermediate time and scaling factor) via a 9k-parameter predictor trained by distillation, plus AMED-Plugin for existing solvers, reaching strong FIDs at about 5 NFE.
Problem
Fast ODE-based sampling of diffusion models degrades sharply at extremely small numbers of function evaluations (around 5 NFE), due to truncation errors of numerical solvers.
Approach
Observes that PF-ODE sampling trajectories nearly lie in a two-dimensional subspace, and uses the mean value theorem to approximate the integral with one learned intermediate evaluation point and scaling factor per step, predicted by a tiny network trained via distillation from a teacher trajectory; the same idea is applied as a plugin to other solvers.
Key results
- With 5 NFE, AMED-Plugin on iPNDM reaches 6.61 FID on CIFAR-10 unconditional generation.
- AMED-Solver reaches 10.74 FID on ImageNet 64x64 at 5 NFE (conditional) and 13.20 FID on LSUN Bedroom 256x256 at 5 NFE.
- On Stable Diffusion, AMED-Plugin on DPM-Solver++(2M) improves FID over the baseline at 8 to 20 NFE (e.g., 13.24 vs 14.58 at 20 NFE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AMED-Solver's learned mean-direction step | single-step ODE sampling · CIFAR10 32×32, 5 NFE | FID: 57.30 -> 7.59 | Helped |
| add the AMED predictor to iPNDM | iPNDM · CIFAR10 32×32, 5 NFE | FID: 13.59 -> 6.61 | Helped |
| add AMED-Solver's learned mean-direction step | single-step ODE sampling · ImageNet 64×64, 5 NFE | FID: 42.41 -> 10.74 | Helped |
| add the AMED predictor to iPNDM | iPNDM · ImageNet 64×64, 5 NFE | FID: 18.99 -> 13.83 | Helped |
| add AMED-Solver's learned mean-direction step | single-step ODE sampling · FFHQ 64×64, 5 NFE | FID: 87.10 -> 14.80 | Helped |
| add the AMED predictor to iPNDM | iPNDM · FFHQ 64×64, 5 NFE | FID: 17.17 -> 12.49 | Helped |
| add AMED-Solver's learned mean-direction step | single-step ODE sampling · LSUN Bedroom 256×256, 5 NFE | FID: 80.60 -> 13.20 | Helped |
| add the AMED predictor to DPM-Solver-2 | DPM-Solver-2 · LSUN Bedroom 256×256, 5 NFE | FID: 80.60 -> 25.68 | Helped |
| add the AMED predictor to DPM-Solver++(2M) | DPM-Solver++(2M) · Stable-Diffusion, 8 NFE | FID: 21.33 -> 18.92 | Helped |
| use EDM instead of DPM-Solver++(3M) as AMED-Solver's teacher (ablation) | AMED-Solver · CIFAR10 32×32, 5 NFE | FID: 12.34 -> 7.59 | Helped |
| use iPNDM instead of DPM-Solver++(3M) as the AMED-Plugin teacher (ablation) | AMED-Plugin on iPNDM · CIFAR10 32×32, 5 NFE | FID: 7.57 -> 6.61 | Helped |
| use a logSNR schedule instead of a uniform schedule (ablation) | DPM-Solver++(3M) · CIFAR10 32×32, 5 NFE | FID: 26.90 -> 24.97 | Helped |
Surprising findings
- The paper reports that iPNDM achieves very strong results and outperforms other ODE solvers in many cases.
- Distillation-free solver-based AMED reports that analytical first step yields little degradation or even improvement for low-resolution datasets.
Limitations
- Fast ODE solvers remain highly sensitive to time schedules at small NFE, and AMED-Plugin only partly alleviates this.
- Performance on LSUN Bedroom for AMED-Plugin is inconsistent across solvers, e.g., worse than DPM-Solver-2 baseline at some NFE values.
Experimental setup
Datasets: CIFAR-10 FFHQ 64x64 ImageNet 64x64 LSUN Bedroom MS-COCO
Benchmarks: CIFAR-10 ImageNet 64x64 FFHQ 64x64 LSUN Bedroom MS-COCO
Metrics: FID
Baselines: DDIM EDM DPM-Solver-2 DPM-Solver++(3M) UniPC iPNDM
Builds on: DPM-Solver DDIM EDM iPNDM DPM-Solver++
Closest papers by meaning
- Distilling ODE Solvers of Diffusion Models into Smaller Steps (CVPR 2024 · similarity 0.91)
- On Accelerating Diffusion-Based Sampling Processes via Improved Integration Approximation (ICLR 2024 · similarity 0.90)
- Accelerating Diffusion Sampling with Optimized Time Steps (CVPR 2024 · similarity 0.90)
- Differentiable Solver Search for Fast Diffusion Sampling (ICML 2025 · similarity 0.90)
- MaRS: A Fast Sampler for Mean Reverting Diffusion based on ODE and SDE Solvers (ICLR 2025 · similarity 0.89)
- TADA: Improved Diffusion Sampling with Training-free Augmented DynAmics (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.