Research map / Optimization / Nonconvex and smooth optimization
Accelerating Convergence of Score-Based Diffusion Models, Provably
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes training-free accelerated DDIM-type (deterministic) and DDPM-type (stochastic) samplers with provable convergence rates O(1/T^2) and O(1/T), using higher-order approximation ideas akin to DPM-Solver-2.
Problem
Score-based diffusion samplers need many score evaluations; existing DDIM/DDPM convergence theory is slow (O(1/T) and O(1/sqrt(T))) and training-free accelerated samplers lack non-asymptotic guarantees.
Approach
A second-order (higher-order approximation) update that adds a momentum-like correction term reusing the previous score evaluation for the deterministic sampler, and an injected-noise two-step update for the stochastic sampler derived from a higher-order expansion of the conditional density; both are training-free and need only l2 score accuracy.
Key results
- Accelerated deterministic sampler achieves TV bound O(d^6 log^6 T / T^2) plus score and Jacobian error terms, improving DDIM's O(1/T) rate; iteration complexity poly(d)/sqrt(eps).
- Accelerated stochastic sampler achieves TV bound O(d^3 log^4.5 T / T) plus sqrt(d) score error term, improving DDPM's O(1/sqrt(T)) rate.
- On LSUN-Churches, CelebA-HQ and LSUN-Bedroom at 5 NFEs, the accelerated DDIM-type sampler yields crisper, less noisy images than vanilla DDIM (qualitative only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add midpoint prediction and a momentum-style score-difference correction to DDIM | DDIM · TV distance convergence | convergence rate: O(\frac{1}{T}) -> O(\frac{1}{{T}^2}) | Helped |
| Add a noise-injected intermediate point before the DDPM-style update | DDPM · TV distance convergence | convergence rate: O(\frac{1}{\sqrt{T}}) -> O(\frac{1}{T}) | Helped |
| Use the accelerated deterministic sampler instead of DDIM | DDIM-type sampler · CelebA-HQ, LSUN-Bedroom, LSUN-Churches (5 NFEs) | sample image quality | Helped |
Surprising findings
- The deterministic accelerated sampler requires an additional Jacobian-error assumption; with only score-error accuracy, the deterministic sampler may fail to achieve small TV error.
Limitations
- Theory's dimension dependency is sub-optimal (poly(d) with large powers).
- Experiments are qualitative only, with no quantitative FID-type comparison; stochastic sampler not evaluated empirically.
- Deterministic theory requires Jacobian accuracy assumption in addition to score accuracy.
Experimental setup
Datasets: CelebA-HQ LSUN-Bedroom LSUN-Churches
Benchmarks: CelebA-HQ LSUN-Bedroom LSUN-Churches
Metrics: TV distance bound (theoretical) qualitative sample quality vs NFE
Baselines: DDIM DDPM
Builds on: DDIM DDPM DPM-Solver-2 probability flow ODE
Closest papers by meaning
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.93)
- Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers (ICLR 2025 · similarity 0.93)
- Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach (ICLR 2025 · similarity 0.92)
- Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Acceleration (NeurIPS 2025 · similarity 0.92)
- A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models (ICML 2024 · similarity 0.91)
- A Score-Based Deterministic Diffusion Algorithm with Smooth Scores for General Distributions (AAAI 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.