Research map / Image generation / Fast diffusion sampling
Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that DDPM with a specific coefficient design (eta_t = 1-alpha_t, sigma_t^2 as given) adapts to unknown low-dimensional structure, achieving TV error O(k^2/sqrt(T)) plus score error, and that this design is essentially unique in avoiding ambient-dimension error per step.
Problem
Understanding convergence of the DDPM sampler when the target distribution lies near a low-dimensional structure; existing bounds scale polynomially with ambient dimension d.
Approach
A deterministic-style analysis of the DDPM reverse dynamics with a specific coefficient schedule (eta_t* = 1-alpha_t, sigma_t*^2 as defined) that yields discretization error depending on intrinsic dimension k (via covering number) rather than ambient dimension d, and a lower bound showing other coefficient choices incur error linear in d per step.
Key results
- Theorem 1: TV(q_1, p_1) <= C (k+log d)^2 log^3 T / sqrt(T) + C eps_score log T under the proposed coefficients, giving iteration complexity k^4/eps^2.
- Theorem 2: for Gaussian N(0,I_k) target, any other coefficients incur per-step KL error at least (d/4)(eta_t-eta_t*)^2 + (d/40)(sigma*^2/sigma^2-1)^2, linear in d.
- Simulation with k=8 shows the proposed design gives dimension-independent error while eta=sigma^2=1-alpha_t grows with d.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the coefficient design η_t = η_t* and σ_t = σ_t* instead of η_t = σ_t² = 1 − α_t (ablation) | DDPM with exact scores · Gaussian target N(0,I_k), k=8; d from 10 to 1000; T ∈ {100,200,500,1000} | TV distance | Helped |
| Use the coefficient design η_t = η_t* and σ_t = σ_t* instead of η_t = σ_t² = 1 − α_t (ablation) | DDPM with exact scores · Gaussian target N(0,I_k), k=8; d from 10 to 1000; T ∈ {100,200,500,1000} | KL divergence | Helped |
| Use arbitrary coefficients that differ from η_t* or σ_t*² (ablation) | DDPM with perfect score estimation · Gaussian target N(0,I_k), k ≤ d/2; each denoising step | Expected conditional KL divergence: ≥ d/4(η_t − η_t*)² + d/40(σ_t*²/σ_t² − 1)² | Hurt |
Surprising findings
- The widely used coefficient choice eta_t = sigma_t^2 = 1-alpha_t incurs error growing with ambient dimension even when the target is intrinsically low-dimensional.
Limitations
- Quartic dependence on intrinsic dimension k is likely suboptimal.
- Uniqueness of the coefficient design is not established; lower bound applies only to the upper-bound decomposition.
- Lower bound analysis is for Gaussian N(0,I_k) target, and the simulation uses only this Gaussian example.
Experimental setup
Metrics: total variation distance KL divergence
Baselines: DDPM with eta_t = sigma_t^2 = 1-alpha_t coefficients
Builds on: DDPM Chen et al. 2022 score-based sampler convergence analysis
Closest papers by meaning
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.93)
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.92)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.92)
- Beyond Scores: Proximal Diffusion Models (NeurIPS 2025 · similarity 0.91)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.91)
- Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.