Research map / Image generation / Fast diffusion sampling
Provable Efficiency of Guidance in Diffusion Models for General Data Distribution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that classifier-free guidance in continuous-time diffusion models decreases the expected reciprocal of classifier probability for general data distributions, with discretization and score-error robustness bounds, validated on a 1D GMM and ImageNet.
Problem
Existing theory of diffusion guidance only covers special cases (isotropic Gaussian or 1D compactly supported classes); it is unknown why guidance improves sample quality under general data distributions.
Approach
Shows that the reciprocal of the classifier probability, as a function of the noisy state, is a martingale under the unguided reverse process; using Ito's formula, the guided reverse SDE then yields a decrease in its expectation proportional to the squared gap between conditional and unconditional scores, so guidance improves the average (not uniform) sample quality.
Key results
- Guidance does not uniformly improve classifier probability: P(p(1|Y^w) >= p(1|Y^0)) stays below 1 for all w<10 on ImageNet and GMM experiments.
- Expected negative reciprocal classifier probability increases (quality improves) with guidance scale w in both GMM and ImageNet experiments.
- Discretization analysis gives KL bound scaling as O(log^3 N / N) plus score-error term (1+w^2) eps^2 log N.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the guidance scale (ablation) | Diffusion sampling on ImageNet · ImageNet | average negative reciprocal classifier probability | Helped |
| Add non-zero guidance | Diffusion sampling under general data distributions · General data distributions | average reciprocal classifier probability | Helped |
| Increase the guidance scale to w<10 (ablation) | Diffusion sampling on ImageNet · ImageNet | probability of improved per-sample classifier probability | Mixed |
| Increase guidance scale across the tested values (ablation) | Discrete diffusion sampling on ImageNet · ImageNet, TV=0.30 and TV=0.10 | relative error ratio: 0.447 -> 0.002; 0.440 -> 0.002 | Helped |
| Use large guidance scale | Diffusion sampling · Real-world applications | sample diversity | Hurt |
Surprising findings
- Guidance does not uniformly improve sample quality: a nontrivial fraction of samples get lower classifier probability even when the average improves.
- Large guidance weights can reduce sample diversity, a known drawback not captured by the metric.
Limitations
- Metric is expected reciprocal classifier probability, not Inception Score; extension to IS left as future work.
- Analysis uses the true conditional probability rather than an estimated classifier.
- Discretization and estimation bounds are not tight, per the authors.
Experimental setup
Datasets: ImageNet synthetic 1D Gaussian mixture
Benchmarks: ImageNet 1D Gaussian mixture model
Metrics: probability of improved classifier probability expected negative reciprocal classifier probability
Baselines: unguided diffusion (w=0)
Builds on: classifier-free guidance DDPM Inception Score
Closest papers by meaning
- Theoretical insights for diffusion guidance: A case study for Gaussian mixture models (ICML 2024 · similarity 0.93)
- What does guidance do? A fine-grained analysis in a simple setting (NeurIPS 2024 · similarity 0.92)
- Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Gradient Guidance for Diffusion Models: An Optimization Perspective (NeurIPS 2024 · similarity 0.90)
- Don't Play Favorites: Minority Guidance for Diffusion Models (ICLR 2024 · similarity 0.89)
- Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.