Research map / Optimization / Nonconvex and smooth optimization
Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Acceleration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes Wasserstein-2 convergence of score-based diffusion models under Euler, exponential integrator, and randomized-midpoint discretizations, and proposes a Hessian-based second-order local-linearization sampler achieving O~(1/ε) iteration complexity versus O~(1/ε²) for first-order schemes.
Problem
Wasserstein convergence guarantees for score-based diffusion models are underexplored, especially how discretization schemes affect them and whether second-order (Hessian) information can accelerate sampling in W2 distance.
Approach
Derives W2 bounds for EM, EI, randomized midpoint, and a combined exponential-integrator-randomized-midpoint scheme under strong log-concavity, and introduces a second-order sampler using local linearization of the score (requiring Hessian and third-order derivative estimates) that improves the iteration complexity from 1/ε² to 1/ε.
Key results
- Euler, exponential integrator, and randomized-midpoint schemes all achieve W2 error rates of order 1/ε² iterations (N = O(ε^-2 log(1/ε)) with h = O(ε²)) under strong log-concavity.
- The Hessian-based second-order sampler (SO) achieves N = O~(1/ε) iterations with step size h = O(ε), improving on the 1/ε² rate of vanilla diffusion sampling.
- Numerical experiments on penalized logistic regression posteriors show the SO sampler outperforms EM, EI, REM, and REI in W1 error along the first dimension for a fixed N = 10/h.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use local linearization with Hessian and higher-order score information instead of first-order discretization | Score-based diffusion model · Wasserstein-2 convergence under stated assumptions | iterations: \widetilde{\mathcal{O}}(1/\varepsilon^2) -> \widetilde{\mathcal{O}}(1/\varepsilon) | Helped |
| Use randomized midpoint evaluation rather than Euler or exponential-integrator evaluation | Score-based diffusion model · Wasserstein-2 convergence under stated assumptions | convergence order | No clear effect |
| Combine randomized midpoint evaluation with the exponential integrator | Score-based diffusion model · Wasserstein-2 convergence under stated assumptions | convergence rate | No clear effect |
| Use the Hessian-based second-order sampler instead of EM, EI, REM, or REI | Score-based diffusion model · Penalized logistic regression posterior | Wasserstein distance | Helped |
Surprising findings
- The randomized midpoint method does not improve the convergence order over EM and EI in the W2 setting, because the extra variance term from random intermediate points obscures the gain, though it enables parallel computation.
Limitations
- Requires strong log-concavity of the data distribution and Lipschitz score assumptions, which are restrictive.
- Restricted to the Ornstein-Uhlenbeck forward process with f=-x/2, g=1 and SDE-based samplers only.
- Second-order sampler assumes accurate estimates of Hessian and higher-order (time and spatial) score derivatives.
Experimental setup
Metrics: Wasserstein distance
Baselines: Euler-Maruyama scheme exponential integrator vanilla randomized midpoint method exponential integrator with randomized midpoint
Builds on: randomized midpoint method exponential integrator local linearization method
Closest papers by meaning
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.92)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
- Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis (ICLR 2025 · similarity 0.89)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.89)
- Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers (ICLR 2025 · similarity 0.89)
- Wasserstein Convergence of Critically Damped Langevin Diffusions (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.