Research map / Image generation / Fast diffusion sampling
Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives discrete-time, elementary (SDE/ODE-free) non-asymptotic TV convergence bounds for a probability-flow-ODE deterministic sampler (O(1/T)) and a DDPM-type stochastic sampler (O(1/sqrt T)) under l2 score errors, plus accelerated variants reaching O(1/T^2) and O(1/T).
Problem
Theoretical convergence guarantees for diffusion models' sampling phase in discrete time, especially for the deterministic probability-flow-ODE (DDIM-like) sampler, with explicit dependence on score estimation error and without strong smoothness assumptions on the data.
Approach
Analyze discrete-time sampler updates directly with an elementary non-asymptotic density-ratio argument (tracking typical points and cumulative score errors) instead of continuous-time SDE/ODE tools, yielding improved 1/T rates for the ODE sampler and accelerated samplers using extra estimated Jacobian-related/correction quantities.
Key results
- Deterministic probability-flow ODE sampler achieves TV(q1,p1) bounded by O(d^2 log^4 T / T) plus score-error terms, i.e., iteration complexity about d^2/eps, improving on prior ODE analyses.
- DDPM-type stochastic sampler achieves TV <= C d^2 log^3 T / sqrt(T) + C sqrt(d) eps_score log^2 T, matching the 1/sqrt(T) state of the art.
- Accelerated deterministic sampler improves complexity to O(d^3/sqrt(eps)) and accelerated stochastic sampler to O(d^3/eps).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add higher-order bias correction and estimates w_t to the probability flow ODE sampler (ablation) | Probability flow ODE sampler · Forward-process q1 vs generated p1 | TV distance convergence scaling in accuracy ε: \widetilde{O}(1/\varepsilon) -> \widetilde{O}(1/\sqrt{\varepsilon}) | Helped |
| Add correction term using estimates v_t to the DDPM-type stochastic sampler (ablation) | DDPM-type stochastic sampler · Forward-process q1 vs generated p1 | TV distance convergence scaling in accuracy ε: \widetilde{O}(1/\varepsilon^2) -> \widetilde{O}(1/\varepsilon) | Helped |
Surprising findings
- The plain probability flow ODE sampler's TV distance may not improve as score error decreases; a counterexample shows TV=1 for arbitrarily small L2 score error, so an extra Jacobian-error assumption is needed.
Limitations
- Suboptimal dimension dependence (d^2 or d^3) compared to variants with corrector steps.
- Requires an additional Jacobian-error assumption for the deterministic sampler; L2 score error alone is insufficient for TV guarantees.
- Accelerated variants assume access to additional estimated quantities beyond the score.
Experimental setup
Metrics: total variation distance KL divergence iteration complexity
Baselines: Chen et al. 2022 SDE-based DDPM analysis Chen et al. 2023 probability flow ODE analysis Lee et al. 2022 polynomial convergence Lee et al. 2023 probability flow ODE corrector variants
Builds on: DDPM DDIM probability flow ODE score-based generative modeling
Closest papers by meaning
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.93)
- Evaluating the design space of diffusion-based generative models (NeurIPS 2024 · similarity 0.91)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.91)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.90)
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.