Research map / Image generation / Fast diffusion sampling
Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RX-DPM, a training-free ODE sampler for diffusion models that applies Richardson-style extrapolation between single-step and multi-step solutions on arbitrary time grids, improving sample quality at fixed NFEs.
Problem
Diffusion probabilistic models are slow to sample because they need many iterative ODE/SDE solver steps; numerical truncation error degrades sample quality at low NFE.
Approach
Applies Richardson extrapolation every k steps by combining a one-step and a k-step ODE solution from the same time point, with coefficients derived for arbitrary (non-uniform) time schedules from the local truncation error of the baseline solver; the extra solution reuses cached network outputs so NFEs are unchanged.
Key results
- On CIFAR-10 with DPM-Solver-2 at 10 NFEs, RX-DPM-Solver-2 reduces FID from 15.06 to 12.94 (and on ImageNet 64x64 DPM-Solver-3 at 9 NFEs reaches FID 7.08 vs 7.45).
- On Stable Diffusion V2 with DDIM at 15 NFEs, RX-DDIM improves FID from 19.15 to 17.24, and at 50 NFEs from 18.65 to 17.83.
- RX-Euler adds negligible runtime overhead (e.g., 6.436s to 6.443s per batch on ImageNet 64x64 with EDM) and no extra NFEs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply grid-aware Richardson extrapolation to DDIM | DDIM / Stable Diffusion V2 · COCO2014 val prompts, 15 NFEs | FID: 19.15 -> 17.24 | Helped |
| Apply grid-aware Richardson extrapolation to DDIM | DDIM / Stable Diffusion V2 · COCO2014 val prompts, 15 NFEs | CLIP: 31.727 -> 31.629 | Hurt |
| Apply grid-aware Richardson extrapolation to DDIM | DDIM / Stable Diffusion V2 · COCO2014 val prompts, 20 NFEs | FID: 18.43 -> 17.12 | Helped |
| Apply grid-aware Richardson extrapolation to DDIM | DDIM / Stable Diffusion V2 · COCO2014 val prompts, 30 NFEs | FID: 19.00 -> 17.62 | Helped |
| Apply grid-aware Richardson extrapolation to DDIM | DDIM / Stable Diffusion V2 · COCO2014 val prompts, 50 NFEs | FID: 18.65 -> 17.83 | Helped |
| Apply RX-DPM to DPM-Solver-2 | DPM-Solver-2 · CIFAR-10, 10 NFEs | FID: 15.06 -> 12.94 | Helped |
| Apply RX-DPM to DPM-Solver-2 | DPM-Solver-2 · LSUN Bedroom, 15 NFEs | FID: 6.44 -> 5.72 | Helped |
| Apply RX-DPM to DPM-Solver-3 | DPM-Solver-3 · LSUN Bedroom, 15 NFEs | FID: 4.04 -> 4.04 | No clear effect |
| Apply RX-DPM to F-PNDM | F-PNDM · LSUN Church | FID: 8.69 -> 8.85; 9.13 -> 9.41 | Hurt |
| Apply RX-DPM to S-PNDM | S-PNDM · CIFAR-10, 5 steps | FID: 18.3 -> 19.69 | Hurt |
| Use arbitrary-grid-aware coefficients instead of fixed Richardson coefficients (ablation) | RX-Euler · EDM backbone, across tested datasets and NFEs | FID | Helped |
| Increase extrapolation interval from k=2 to k=3 or k=4 (ablation) | RX-Euler · EDM backbone, varied NFEs | FID | Mixed |
Surprising findings
- RX-DDIM yields lower CLIP scores at NFE=15 on Stable Diffusion V2, which the authors attribute to classifier-guidance scale tuned for DDIM.
- RX-DPM does not clearly help F-PNDM on LSUN Church, because the baseline itself degrades with more steps under finer discretization.
Limitations
- Designed primarily for ODE solvers; integration with SDE/stochastic sampling is partial and its benefit is sometimes offset.
- Higher-order extension relies on an assumed linear error propagation that does not hold in general.
- Heun's method beats RX-Euler at larger NFEs, and no clear advantage for F-PNDM on LSUN Church.
Experimental setup
Datasets: CIFAR-10 FFHQ AFHQv2 ImageNet CelebA LSUN Bedroom LSUN Church COCO2014
Benchmarks: CIFAR-10 FFHQ AFHQv2 ImageNet CelebA LSUN Bedroom LSUN Church COCO2014
Metrics: FID CLIP score
Baselines: Euler method Heun's method LA-DPM IIA DDIM DPM-Solver DEIS PNDM NPR-DPM SN-DPM
Builds on: Richardson extrapolation DDIM DPM-Solver PNDM EDM
Closest papers by meaning
- Accelerating Diffusion Sampling with Optimized Time Steps (CVPR 2024 · similarity 0.92)
- A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models (ICLR 2024 · similarity 0.91)
- Learning to Discretize Denoising Diffusion ODEs (ICLR 2025 · similarity 0.91)
- On Accelerating Diffusion-Based Sampling Processes via Improved Integration Approximation (ICLR 2024 · similarity 0.90)
- PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores (ICLR 2025 · similarity 0.90)
- Diffusion Sampling Correction via Approximately 10 Parameters (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.