Research map / Image generation / Fast diffusion sampling
Learning to Integrate Diffusion ODEs by Averaging the Derivatives
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes secant losses that learn the average velocity (secant) of diffusion PF-ODEs via Monte Carlo integration and Picard iteration, enabling few-step generation by fine-tuning or distilling EDM and SiT.
Problem
Slow diffusion inference: numerical ODE solvers degrade badly below ~10 steps, while distillation methods are costly, complex, and unstable.
Approach
Model the secant (average of tangents/velocities between two times) with a neural network, trained with integral-derived losses that use Monte Carlo sampling of an intermediate time and Picard-style estimation of the unobserved point; the regression targets are the diffusion target or the teacher velocity, avoiding explicit derivatives used by consistency models.
Key results
- SDEI secant version of EDM reaches 2.14 FID on CIFAR-10 with 10 steps.
- Secant version of SiT-XL/2 with guidance interval reaches 2.27 FID at 4 steps and 1.96 FID at 8 steps on ImageNet-256.
- Without guidance interval, STEE reaches 2.33 FID at 8 steps and STEI reaches 2.78 at 4 steps on ImageNet-256.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add guidance interval to STEI | STEI · ImageNet-256×256, 4 steps | FID: 2.78 -> 2.27 | Helped |
| add guidance interval to STEI | STEI · ImageNet-256×256, 8 steps | FID: 2.36 -> 1.96 | Helped |
| add guidance interval to STEE | STEE · ImageNet-256×256, 4 steps | FID: 3.02 -> 2.55 | Helped |
| add guidance interval to STEE | STEE · ImageNet-256×256, 8 steps | FID: 2.33 -> 1.96 | Helped |
| set λ from 0.1 to 1.0 (ablation) | STEI · ImageNet-256×256, 4 steps | FID: 3.96 -> 2.84 | Helped |
| set λ from 1.0 to 2.0 (ablation) | STEI · ImageNet-256×256, 4 steps | FID: 2.84 -> 3.96 | Hurt |
| use discrete, generation-only t and s sampling instead of adding continuous sampling or inversion capability (ablation) | SDEI and STEI · ImageNet-256×256 | FID | Hurt |
| sample r uniformly instead of using biased sampling toward t, s, or the midpoint (ablation) | secant losses · CIFAR-10 and ImageNet-256×256 | FID | Helped |
| use interior-point estimation rather than end-point estimation (ablation) | secant losses · CIFAR-10 and ImageNet-256×256 | FID | Helped |
Surprising findings
- Consistency-model training diverged on ImageNet-256 in the authors' experiments, while secant losses trained stably.
- Distillation variants generally outperform training variants, and interior-point estimation degrades more slowly as steps decrease.
Limitations
- Significant performance gap between 1-step and 8-step generation.
- ImageNet performance relies heavily on classifier-free guidance; theoretical link to CFG unexplored.
- Requires training data, which may be limiting in data-scarce settings.
Experimental setup
Datasets: CIFAR-10 ImageNet-256x256
Benchmarks: CIFAR-10 ImageNet-256x256
Metrics: FID IS
Baselines: DPM-Solver DPM-Solver++ UniPC DEIS LD3 Flow-DPM-Solver Consistency Training/Distillation (iCT) IMM Shortcut Models CTM MeanFlow
Builds on: EDM SiT consistency models classifier-free guidance guidance interval
Closest papers by meaning
- On Accelerating Diffusion-Based Sampling Processes via Improved Integration Approximation (ICLR 2024 · similarity 0.86)
- Distilling ODE Solvers of Diffusion Models into Smaller Steps (CVPR 2024 · similarity 0.86)
- S4S: Solving for a Fast Diffusion Model Solver (ICML 2025 · similarity 0.85)
- Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration (NeurIPS 2025 · similarity 0.85)
- Lipschitz Singularities in Diffusion Models (ICLR 2024 · similarity 0.85)
- Physics-Informed Diffusion Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.