Research map / Image generation / Fast diffusion sampling

Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models

AAAI 2025 · Image generation › Fast diffusion sampling · approach: Guidance for conditional diffusion · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Searches per-step guidance policies with differentiable NAS and proposes Adaptive Guidance (AG), which drops CFG once conditional and unconditional predictions converge (~25% NFE savings), plus LinearAg, which replaces unconditional evaluations with affine combinations of past iterates.

Problem

Classifier-free guidance doubles the number of function evaluations per denoising step in text-conditioned diffusion models, increasing sampling latency; the default of applying CFG at every step may be suboptimal.

Approach

Casts per-step guidance choice (unconditional, conditional, or CFG with several scales) as a differentiable NAS problem with softmax-relaxed alphas, observes that conditional and unconditional predictions become nearly aligned in late steps, and uses this to switch off CFG adaptively via a cosine-similarity threshold; additionally estimates unconditional scores by least-squares affine combinations

Key results

• Adaptive Guidance (gamma=0.991) on EMU-768 uses about 29.6 NFEs vs 40 for CFG, with human votes 498 (AG) vs 502 (CFG), no significant difference (p=0.603).
• AG replicates CFG outputs more accurately than naive step reduction at equal NFE budgets on LDM-512 (SSIM comparison).
• LinearAg, which replaces unconditional evaluations with linear combinations of past iterates, reaches about 75% guidance NFE savings per the conclusion, at the cost of not replicating the baseline one-to-one.

Measured effects reported in the paper

Surprising findings

• Cosine similarity between conditional and unconditional predictions increases almost monotonically over time, reaching near-perfect alignment late in denoising, so late-step CFG is redundant.
• Unconditional noise predictions can be estimated with high accuracy from a few scalar-weighted past iterates via least squares, so entire network evaluations can be replaced.

Limitations

• LinearAg no longer replicates the baseline one-to-one and is presented as a proof of concept; errors accumulate autoregressively.
• Search is performed only on a 900M-parameter model trained from scratch on a commissioned dataset; generalization shown on a single larger model (EMU-768).
• Human evaluation uses a single comparison setting; reductions of NFEs rely on a single threshold hyperparameter.

Experimental setup

Datasets: CC3M Open User Input (OUI) prompts

Benchmarks: OUI prompts (1,000 subset)

Metrics: SSIM human preference win/lose votes NFEs Wilcoxon signed-rank test

Baselines: Classifier-free guidance (20 steps, 40 NFEs) naive step reduction guidance distillation AutoDiffusion

Builds on: classifier-free guidance DARTS Stable Diffusion InstructPix2Pix DPM-Solver++

Closest papers by meaning

• Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.93)
• Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models (ICLR 2025 · similarity 0.93)
• Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.92)
• Towards a Golden Classifier-Free Guidance Path via Foresight Fixed Point Iterations (NeurIPS 2025 · similarity 0.92)
• Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance (CVPR 2024 · similarity 0.92)
• No Training, No Problem: Rethinking Classifier-Free Guidance for Diffusion Models (ICLR 2025 · similarity 0.92)