Research map / Image generation / Fast diffusion sampling
On Error Propagation of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Develops a theoretical framework for error propagation in diffusion models (modular error, cumulative error, propagation equation) and uses a bootstrapped upper bound of cumulative error (MMD-based) as a regularizer, improving DDPM FID on CIFAR-10, ImageNet and CelebA.
Problem
Diffusion models are sequential chains that may suffer error propagation (related to exposure bias), but it is unclear whether and why this affects them, and how to reduce it.
Approach
Formalizes modular and cumulative KL errors and a propagation inequality E_t^cumu >= E_{t+1}^cumu + E_t^mod showing amplification factor >= 1 in standard DMs; regularizes training with an MMD-based upper bound of the cumulative error, estimated efficiently via bootstrapped (TD-style) warm-started backward chains.
Key results
- Regularized DDPM reaches FID 2.93 on CIFAR-10, 2.55 on ImageNet and 1.22 on CelebA, versus 3.61, 3.62 and 1.73 for vanilla DDPM.
- Relative FID reductions reported as 18.83% (CIFAR-10), 29.55% (ImageNet) and 29.47% (CelebA) over DDPM.
- Ablations show removing the exponential loss-weight schedule raises CIFAR-10 FID from 2.93 to 3.12, and random warm-start initialization degrades FID to 3.38 at L=5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add cumulative-error regularization with bootstrapping | DDPM · CIFAR-10 | FID: 3.61 -> 2.93 | Helped |
| Add cumulative-error regularization with bootstrapping | DDPM · ImageNet | FID: 3.62 -> 2.55 | Helped |
| Add cumulative-error regularization with bootstrapping | DDPM · CelebA | FID: 1.73 -> 1.22 | Helped |
| Add cumulative-error regularization instead of ADM-IP input perturbation | DDPM · CIFAR-10 | FID: 3.25 -> 2.93 | Helped |
| Add cumulative-error regularization instead of ADM-IP input perturbation | DDPM · ImageNet | FID: 2.72 -> 2.55 | Helped |
| Add cumulative-error regularization instead of ADM-IP input perturbation | DDPM · CelebA | FID: 1.31 -> 1.22 | Helped |
| Add an exponential loss-weight schedule rather than equal weights (ablation) | DDPM w/ proposed regularization · CIFAR-10 | FID: 3.12 -> 2.93 | Helped |
| Add an exponential loss-weight schedule rather than equal weights (ablation) | DDPM w/ proposed regularization · CelebA | FID: 1.35 -> 1.22 | Helped |
| Add warm-start initialization with L=5 (ablation) | DDPM w/ proposed regularization · CIFAR-10 | FID: 3.38 -> 2.93 | Helped |
| Add warm-start initialization with L=5 (ablation) | DDPM w/ proposed regularization · CelebA | FID: 1.45 -> 1.22 | Helped |
| Add warm-start initialization with L=7 (ablation) | DDPM w/ proposed regularization · CIFAR-10 | FID: 3.15 -> 2.93 | Helped |
| Add warm-start initialization with L=7 (ablation) | DDPM w/ proposed regularization · CelebA | FID: 1.27 -> 1.22 | Helped |
Surprising findings
- The paper's bootstrapped regularizer with L=5 outperforms the ADM-IP baseline even though ADM is a stronger backbone than DDPM.
Limitations
- Theory relies on strong assumptions, e.g., the noise predictor output is standard Gaussian regardless of input and entropy is non-increasing.
- Bootstrapped estimate is biased due to the approximate warm-start initialization; accuracy depends on sampling length L.
- Evaluated only on small-resolution image datasets with FID as the sole metric.
Experimental setup
Datasets: CIFAR-10 ImageNet 32x32 CelebA 64x64
Benchmarks: CIFAR-10 ImageNet CelebA
Metrics: FID
Baselines: ADM-IP Consistent DM FP-Diffusion DDPM
Builds on: DDPM ADM exposure bias analysis
Closest papers by meaning
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.90)
- Optimizing Quantized Diffusion Models via Distillation with Cross-Timestep Error Correction (AAAI 2025 · similarity 0.90)
- Evaluating the design space of diffusion-based generative models (NeurIPS 2024 · similarity 0.89)
- Slight Corruption in Pre-training Data Makes Better Diffusion Models (NeurIPS 2024 · similarity 0.89)
- EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.89)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.