Research map / Learning theory and algorithms / Sample complexity
Leveraging Drift to Improve Sample Complexity of Variance Exploding Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a drifted variance-exploding SDE forward process with an added drift term that gives exp(-T) forward convergence, yielding the first polynomial sample complexity for VE-based diffusion with reverse SDE under the manifold hypothesis, plus a unified tangent-based analysis covering reverse PFODE.
Problem
Variance-exploding (VE) diffusion models lack efficient theoretical sample-complexity guarantees because their forward process converges to Gaussian noise only at a polynomial 1/poly(T) rate, versus exp(-T) for VP models, and prior VE analyses restrict the diffusion coefficient to be bounded.
Approach
Add a drift term -(1/τ)β_t X_t to the VESDE forward process (τ in [T, T^2]) so the data information decays, giving exp(-T) forward convergence and allowing unbounded β_t (e.g., β_t = t^2); a tangent-based analysis then unifies reverse SDE and PFODE guarantees.
Key results
- Drifted VESDE with beta_t = t^2 and tau = T^2 achieves sample complexity eO(1/(eps_W2^8 eps_TV^2)) under the manifold hypothesis, the first polynomial result for VE-based models with reverse SDE.
- Forward convergence rate of the drifted VESDE is exp(-T)/T^2 for suitable beta_t, versus 1/poly(T) for pure VESDE.
- On CelebA256, drifted VESDE (conservative, no training) gets aesthetic score 5.813 and IS 4.174 versus pure VESDE 5.807 and 4.082.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use conservative drifted VESDE with βt = 1 and τ = T instead of pure VESDE | Pure VESDE trained model · CelebA256 | aesthetic score: 5.807 -> 5.813 | Helped |
| Use conservative drifted VESDE with βt = 1 and τ = T instead of pure VESDE | Pure VESDE trained model · CelebA256 | Inception Score: 4.082 -> 4.174 | Helped |
| Use conservative drifted VESDE with βt = 1 and τ = T instead of pure VESDE | Pure VESDE trained model · Swiss roll | generated distribution uniformity | Helped |
| Use aggressive βt = t² (ablation) | Drifted VESDE · 2-D Gaussian | generated distribution quality | Helped |
| Use conservative βt = t with a small drift term instead of aggressive βt = t² (ablation) | Drifted VESDE · 2-D Gaussian | generated distribution quality | Hurt |
| Remove the drift term (pure VESDE) (ablation) | Drifted VESDE · 2-D Gaussian | generated distribution quality | Hurt |
| Use exponential integrator discretization instead of Euler-Maruyama discretization (ablation) | Drifted VESDE · 2-D Gaussian | generated distribution quality | Helped |
Surprising findings
- A conservative drifted VESDE can improve samples of a pure VESDE model without any retraining.
Limitations
- Theoretical results assume an accurate score; approximated-score error is left to future work for PFODE.
- Reverse PFODE bounds have exponential dependence on R and delta unless an extra Hessian bound assumption is made.
- Real-world experiments are limited to a single CelebA256 comparison with two quantitative metrics.
Experimental setup
Datasets: Swiss roll 2-D Gaussian 1D-GMM CelebA256
Benchmarks: Swiss roll 2-D Gaussian CelebA256
Metrics: aesthetic score Inception Score qualitative sample inspection density uniformity
Baselines: pure VESDE (sigma_t^2 = t, t^2) VPSDE Chen et al. 2023c Lee et al. 2022 Gao et al. 2023
Builds on: VPSDE VESDE Bortoli 2022 manifold sample complexity Chen et al. 2023 Girsanov-based convergence exponential integrator discretization
Closest papers by meaning
- Systematic Hazard Sampling: Minimal-Variance Inference for Discrete Diffusion and Flow Models (NeurIPS 2026 · similarity 0.89)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.88)
- Improved Discretization Complexity Analysis of Consistency Models: Variance Exploding Forward Process and Decay Discretization Scheme (ICML 2025 · similarity 0.87)
- Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers (ICLR 2025 · similarity 0.87)
- Lipschitz Singularities in Diffusion Models (ICLR 2024 · similarity 0.87)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.