Research map / Image generation / Fast diffusion sampling
Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives exact backward SDE and probability-flow ODE solutions for Gaussian data diffusion models, computes exact 2-Wasserstein errors for numerical schemes (EM, EI, DDPM, Euler, Heun, RK4), and tests score approximation on Gaussian microtextures.
Problem
Theoretical understanding of the four error sources (initialization, truncation, discretization, score approximation) in diffusion model sampling, which current bounds (worst-case, Inception/FID-based) do not measure directly in data space.
Approach
Under a centered Gaussian data assumption the score is linear, so the backward SDE and probability flow ODE have explicit solutions and all discretized samplers remain Gaussian; this yields exact 2-Wasserstein errors in data space for each error type and scheme.
Key results
- For the Gaussian CIFAR-10 at NFE=1000 with truncation eps=1e-3, Heun and RK4 reach Wasserstein errors of about 0.17 and 0.17 versus 0.27 for EM.
- Proposition shows Wasserstein(SDE marginals, p_t) <= Wasserstein(ODE marginals, p_t) for all t under exact score, so the SDE sampler is less sensitive to initialization error.
- On the ADSN microtexture, the learned-score EM sampler achieves FID 1.02 while Heun reaches FID 19.4, indicating Heun is more sensitive to score approximation error.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the SDE sampler instead of the ODE sampler with the exact score | Gaussian diffusion sampling · Gaussian distributions | 2-Wasserstein distance to the target | Helped |
| Use Heun or RK4 instead of Euler for the ODE sampler (ablation) | ODE sampling with exact score · CIFAR-10 Gaussian | Wasserstein: 0.26 -> 0.18 (NFE = 500, ε = 10^{-3}, N(0,I)); 0.25 -> 0.19 (NFE = 1000, ε = 10^{-3}, N(0,I)) | Helped |
| Increase NFE from 500 to 1000 (ablation) | EM, EI, DDPM, and Euler samplers · CIFAR-10 Gaussian; ε = 0 | Wasserstein: EM 0.32 -> 0.16; EI 0.30 -> 0.16; DDPM 0.47 -> 0.23; Euler 0.26 -> 0.17 (N(0,I)) | Helped |
| Set truncation time to ε = 10^{-3} rather than ε = 0 (ablation) | EM, EI, DDPM, and Euler samplers · CIFAR-10 Gaussian; NFE = 500 | Wasserstein: EM 0.32 -> 0.40; EI 0.30 -> 0.41; DDPM 0.47 -> 0.53; Euler 0.26 -> 0.32 (N(0,I)) | Hurt |
| Initialize with N(0,I) instead of p_T (ablation) | ODE samplers · CIFAR-10 Gaussian | Wasserstein: Euler 0.20 -> 0.26 (NFE = 500, ε = 0); Euler 0.16 -> 0.18 (NFE = 500, ε = 10^{-3}) | Hurt |
| Initialize with N(0,I) instead of p_T (ablation) | SDE samplers EM, EI, and DDPM · CIFAR-10 Gaussian | Wasserstein: EM 0.32 vs 0.32; EI 0.30 vs 0.30; DDPM 0.47 vs 0.47 (NFE = 500, ε = 0) | No clear effect |
| Use Heun instead of EM with the learned score | Gaussian microtexture diffusion sampling · Gaussian microtexture | empirical Wasserstein: 15.6 -> 56.7 | Hurt |
| Use Heun instead of EM with the learned score | Gaussian microtexture diffusion sampling · Gaussian microtexture | FID: 1.02 -> 19.4\textcolor{white}{8} | Hurt |
| Replace the exact score with a learned score | EM sampler · Gaussian microtexture | empirical Wasserstein: 5.1630±7E-5 -> 15.6 | Hurt |
Surprising findings
- With a learned score, the deterministic Heun scheme produces out-of-distribution samples while stochastic EM is more resilient, contrary to the exact-score analysis favoring Heun.
- Heun and RK4 fail without truncation because the covariance is not stably invertible near t=0.
Limitations
- Results are restricted to Gaussian data distributions, which have limited practical interest.
- Exact analysis does not extend easily to Gaussian mixtures since closed-form solutions and Wasserstein distances are unavailable.
- Learned-score Wasserstein is only approximated because it assumes Gaussianity of the learned sampler outputs.
Experimental setup
Datasets: CIFAR-10 (covariance for Gaussian CIFAR-10) ADSN microtexture from an RGB texture
Benchmarks: CIFAR-10 Gaussian (exact Wasserstein) ADSN microtexture (Wasserstein, FID)
Metrics: 2-Wasserstein distance FID empirical Wasserstein
Baselines: Euler-Maruyama exponential integrator DDPM sampler Euler Heun RK4
Builds on: score-based SDE probability flow ODE DDPM Heun's method
Closest papers by meaning
- Distributional Diffusion Models with Scoring Rules (ICML 2025 · similarity 0.88)
- On Error Propagation of Diffusion Models (ICLR 2024 · similarity 0.88)
- Diffusion Bridge Implicit Models (ICLR 2025 · similarity 0.88)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.88)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.88)
- One-step Diffusion with Distribution Matching Distillation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.