Research map / Image generation / Fast diffusion sampling
Elucidating the Exposure Bias in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes exposure bias in diffusion models by modelling the sampling distribution with prediction error, proposes a metric δ_t, and introduces Epsilon Scaling, a training-free plug-in that scales down the predicted noise at sampling time to improve FID across ADM, DDIM, EDM, LDM, DiT, and PFGM++.
Problem
Exposure bias in diffusion models (input mismatch between training on ground-truth noisy samples and sampling on previously generated samples) lacks in-depth analysis, an explicit metric, and a root-cause explanation.
Approach
Model the practical sampling distribution with a Gaussian approximation of x_0 prediction error, show its variance exceeds the training variance, and fix the resulting drift training-free by scaling down the predicted epsilon by a factor λ(t) (uniform or linear schedule) at sampling time.
Key results
- ADM-ES reaches 2.17 FID on CIFAR-10 with 100-step unconditional sampling, versus 3.37 for ADM, and is reported as state-of-the-art among stochastic samplers.
- On FFHQ 128x128, ADM-ES reduces FID from 14.52 to 6.77 at 100 steps and from 59.35 to 26.14 at 20 steps.
- EDM-ES improves CIFAR-10 unconditional Euler FID at 35 steps from 3.81 to 2.80; LDM-ES improves CelebA-HQ 256 FID at 20 steps from 29.62 to 15.68.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Epsilon Scaling with uniform λ(t) | ADM · CIFAR-10 32×32, 100 steps | FID: 3.37 -> 2.17 | Helped |
| Add Epsilon Scaling with uniform λ(t) | ADM · FFHQ 128×128, 100 steps | FID: 14.52 -> 6.77 | Helped |
| Add Epsilon Scaling | DDIM/DDPM samplers · CelebA 64×64, η=1 (DDPM), 100 steps | FID: 11.33 -> 4.45 | Helped |
| Add Epsilon Scaling | EDM Euler sampler · CIFAR-10, unconditional, 35 steps | FID: 3.81 to 2.80 | Helped |
| Add Epsilon Scaling | LDM DDPM sampler · CelebA-HQ 256×256, 100 steps | FID: 9.31 -> 7.36 | Helped |
| Use linear λ(t)=kt+b instead of uniform λ(t) (ablation) | ADM-ES · CIFAR-10 32×32, 20 steps | FID: 5.15 -> 4.31 | Helped |
Surprising findings
- Optimal variance beta* yields worse FID than the lower-bound variance at 1000 steps, which the authors attribute to exposure bias.
- Epsilon Scaling gives larger FID gains with the first-order Euler sampler than with the Heun sampler, and reduces the advantage of the higher-order solver.
Limitations
- Epsilon Scaling corrects the magnitude of the predicted epsilon but not its direction, so exposure bias is not fully eliminated.
- Scaling schedule requires searching for lambda via trials (6 to 10), and the linear schedule's slope tends to 0 for more steps.
- Gains with DDIM (deterministic sampler) are smaller than with DDPM.
Experimental setup
Datasets: CIFAR-10 LSUN tower FFHQ ImageNet 64x64 ImageNet 128x128 CelebA 64x64 CelebA-HQ 256x256
Benchmarks: CIFAR-10 LSUN tower 64x64 FFHQ 128x128 ImageNet 64x64 ImageNet 128x128 CelebA 64x64 FFHQ 256x256 CelebA-HQ 256x256
Metrics: FID exposure bias delta_t precision recall
Baselines: ADM EDM Improved SDE Restart sampler SA-Solver ADM-IP DDIM LDM PFGM++
Builds on: DDPM DDIM Analytic-DPM EDM Improved DDPM
Closest papers by meaning
- Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps (ICLR 2024 · similarity 0.92)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.90)
- Analyzing and Improving the Training Dynamics of Diffusion Models (CVPR 2024 · similarity 0.90)
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.90)
- Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data (ICML 2024 · similarity 0.90)
- Scaling Inference Time Compute for Diffusion Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.