Research map / Image generation / Fast diffusion sampling

Provably Reliable Classifier Guidance via Cross-Entropy Control

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Guidance for conditional diffusion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that small conditional-KL (cross-entropy) classifier error controls classifier-guidance vector error in diffusion models only under smoothness assumptions, with a counterexample otherwise, and bounds DDPM sampling error accordingly.

Problem

Classifiers for diffusion classifier guidance are trained with cross-entropy (conditional KL), but sampling depends on the L2 accuracy of the guidance gradient; it is unclear whether small cross-entropy implies effective guidance.

Approach

Without smoothness, a vanishing conditional KL can coexist with non-vanishing guidance-gradient MSE (high-frequency perturbations). With bounded support and classifier smoothness matching the true label posteriors, conditional KL eps^2 implies guidance MSE O~(d eps), via integration-by-parts and a reverse log-Sobolev-type argument, and this propagates to a DDPM sampling KL bound.

Key results

• Counterexample: with delta_n=1/n, conditional KL decays to about 2e-7 at n=1000 while guidance MSE stays near 0.341; with delta_n=1/sqrt(n), guidance MSE grows from about 4 to about 330 as n goes 10 to 1000.
• Theorem: under bounded support and classifier smoothness, conditional KL eps^2 implies average guidance MSE of order O~(d eps), and this rate is shown tight by an explicit construction.
• Sampling-error bound: KL between output and target conditional is bounded by score error, guidance error, discretization and early-stopping terms.

Surprising findings

• Small cross-entropy (vanishing conditional KL) does not imply small guidance error in general; guidance MSE can even diverge while KL vanishes.

Limitations

• Requires bounded data support and classifier smoothness assumptions (gradient and Hessian-trace bounds).
• Analysis restricted to guidance strength gamma=1; gamma>1 used in practice is not covered.
• Experiments are small synthetic settings (2D binary classification and GMM), not image diffusion models.

Experimental setup

Datasets: synthetic 2D Gaussian covariates and binary labels synthetic 2D Gaussian mixture model

Metrics: conditional KL divergence guidance MSE

Builds on: classifier guidance DDPM Girsanov-based diffusion sampling analysis Tweedie's formula

Closest papers by meaning

• No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models (ICLR 2025 · similarity 0.89)
• Elucidating the design space of classifier-guided diffusion generation (ICLR 2024 · similarity 0.89)
• TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.88)
• Simple Guidance Mechanisms for Discrete Diffusion Models (ICLR 2025 · similarity 0.88)
• Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.88)
• Provable Efficiency of Guidance in Diffusion Models for General Data Distribution (ICML 2025 · similarity 0.88)