Research map / Image generation / Fast diffusion sampling
What does guidance do? A fine-grained analysis in a simple setting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Rigorously shows that diffusion guidance does not sample the tilted distribution, characterizing its dynamics on mixtures of compactly supported distributions and Gaussian mixtures, and showing that score error plus large guidance pushes samples off-support.
Problem
Diffusion guidance is widely used to sample from a data distribution tilted by a conditional likelihood, but it is unclear what distribution guidance actually samples from and why large guidance weights degrade samples.
Approach
Rigorously prove that the guided probability flow ODE does not sample the tilted distribution and give a fine-grained trajectory analysis on simple mixtures: with large guidance, samples concentrate at the edge of the conditional support (compact case) or drift to infinity at rate sqrt(w) (Gaussian case); arbitrarily small score error can push samples off-support for large w.
Key results
- Compactly supported mixtures: with probability 1-exp(-Omega(w)) guided ODE samples lie within (alpha_2(1-O(1/sqrt(ln w))), alpha_2), i.e. concentrate at the support edge.
- Gaussian mixture 1/2 N(1,1)+1/2 N(-1,1): P(x>=0) >= 1 - exp(-Omega(w^2)) and P(x >= sqrt(w+1)/4) >= 1 - exp(-Omega(w)).
- For any score error epsilon>0, with sufficiently large w (w >= tilde-Omega(sqrt(log log(1/eps)))) the guided ODE with approximate scores yields samples outside the data support, frozen at ln(w)/32.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase guidance strength | Probability flow ODE on synthetic mixture of uniforms · Synthetic mixture of uniforms | sample concentration | Helped |
| Increase guidance beyond w = 3 | Probability flow ODE on synthetic mixture of uniforms · Synthetic mixture of uniforms | sample fidelity | Hurt |
| Use guidance strength w = 3 | Probability flow ODE on synthetic mixture of uniforms · Synthetic mixture of uniforms | sample fidelity | Helped |
| Increase guidance strength | Probability flow ODE on 2-D Gaussian mixture · 2-D Gaussian mixture | positive x-coordinate samples | Helped |
| Increase guidance strength | Probability flow ODE on 2-D Gaussian mixture · 2-D Gaussian mixture | numerical stability | Hurt |
| Use guidance strength s = 5 | ImageNet one-vs-all guidance experiment · ImageNet classes where the paper reports this choice works well | sample quality | Helped |
| Use guidance strength s = 5 | ImageNet one-vs-all guidance experiment · ImageNet classes with support error at s = 5 | support error | Hurt |
| Increase guidance strength | ImageNet one-vs-all guidance experiment · ImageNet basketball class | support error | Mixed |
| Use the direction between positive- and negative-class means | ImageNet guidance experiment · ImageNet basketball class | sample quality | Hurt |
Surprising findings
- Guidance can concentrate samples on the support boundary even though the conditional likelihood is constant over the target support, so it carries no geometric information.
- Arbitrarily small score estimation error can cause guided samples to leave the data support when the guidance weight is large.
Limitations
- Theory restricted to toy settings: two-component mixtures of uniforms or Gaussians in 1D with probability flow ODE.
- Real-data experiments (MNIST, ImageNet) are heuristic and only loosely connected to the assumptions of the theorems.
- Some experiments show numerical instability for large guidance weights.
Experimental setup
Datasets: MNIST ImageNet synthetic mixture of uniforms synthetic mixture of Gaussians
Benchmarks: MNIST one-vs-all ImageNet class-conditional
Metrics: support error (fraction of samples outside class support) sample distortion
Baselines: unguided conditional sampler tilted distribution sampling
Builds on: probability flow ODE classifier-free guidance Wu et al. guidance analysis Bradley and Nakkiran guidance analysis
Closest papers by meaning
- Provable Efficiency of Guidance in Diffusion Models for General Data Distribution (ICML 2025 · similarity 0.92)
- Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Theoretical insights for diffusion guidance: A case study for Gaussian mixture models (ICML 2024 · similarity 0.91)
- Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Gradient Guidance for Diffusion Models: An Optimization Perspective (NeurIPS 2024 · similarity 0.88)
- Diffusion Models without Classifier-free Guidance (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.