Research map / Optimization / Optimizers

Gradient Guidance for Diffusion Models: An Optimization Perspective

NeurIPS 2024 · Optimization › Optimizers · approach: Differentiating through embedded optimization · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies gradient guidance for pre-trained diffusion models from an optimization view, proposing a look-ahead-loss guidance that provably stays in the data's latent subspace and an adaptive fine-tuning variant with O~(1/K) convergence for concave objectives.

Problem

Adapting a pre-trained diffusion model to generate samples that optimize a user-specified differentiable objective while preserving the learned data structure, with theoretical optimization guarantees.

Approach

Replace naive gradient guidance with a look-ahead loss gradient, -beta(t) grad (y - g^T E[x0|xt])^2 using Tweedie's estimate from the pre-trained score; this is provably confined to the data's latent subspace. Iterative guidance (and optional adaptive fine-tuning on self-generated samples) is shown to mimic first-order optimization with regularization from pre-training data.

Key results

• Theory: look-ahead guidance G_loss lies in Span(A) for any data distribution on a latent subspace, whereas naive gradient guidance leaves a nonzero off-subspace mean (Proposition on failure of naive guidance).
• Adaptive fine-tuning variant achieves f* - f(mu_K) = O(d L^2 log K / K) convergence to the unregularized maximum in the subspace for concave smooth f.
• Simulation: G_loss preserves subspace structure better than naive guidance; adaptive fine-tuning reaches the maximal objective value while the non-fine-tuned version converges to a regularized, sub-optimal value.

Measured effects reported in the paper

Surprising findings

• Guidance without fine-tuning cannot reach the global optimum; the pre-trained score acts as a regularizer with lambda >= L, so global optima require updating the score with self-generated samples.

Limitations

• Theory covers Gaussian/linear score classes and concave smooth objectives only.
• Regularization term cannot be made arbitrarily small without fine-tuning, so non-fine-tuned guidance does not reach global optima.
• Image experiments use a synthetic reward model and are only qualitative/illustrative.

Experimental setup

Datasets: ImageNet (reward model pretraining) synthetic subspace data (simulation)

Benchmarks: synthetic subspace simulation image generation reward

Metrics: objective value f off-subspace error convergence gap

Baselines: naive gradient guidance DPS universal guidance

Builds on: DPS classifier guidance Tweedie's formula universal guidance

Closest papers by meaning

• TFG: Unified Training-Free Guidance for Diffusion Models (NeurIPS 2024 · similarity 0.91)
• Theoretical insights for diffusion guidance: A case study for Gaussian mixture models (ICML 2024 · similarity 0.91)
• REG: Rectified Gradient Guidance for Conditional Diffusion Models (ICML 2025 · similarity 0.90)
• Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model (ICLR 2025 · similarity 0.90)
• Provable Efficiency of Guidance in Diffusion Models for General Data Distribution (ICML 2025 · similarity 0.90)
• Variational Control for Guidance in Diffusion Models (ICML 2025 · similarity 0.90)