Research map / Image generation / Fast diffusion sampling

Specificity-Aware Diffusion Steering via Variance-Reduced Sequential Monte Carlo

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives a specificity-aware target from an overlap objective (a powered soft likelihood-ratio gate) and samples it with a variance-reduced local-proposal SMC sampler using fixed-noise JVP-based weight optimization, evaluated on synthetic MoG, CIFAR-10, Stable Diffusion prompts, and pMHC binder design.

Problem

Negative guidance in diffusion models suppresses undesired concepts but also erodes desired mass where desired and undesired distributions overlap, causing mode shift and SMC weight collapse.

Approach

Define the steering target as the desired density restricted to where it dominates the undesired density (overlap-based objective), relax the hard gate via a Bernoulli/sigmoid likelihood-ratio gate with powered reweighting, and build a local forward-backward Gaussian SMC proposal whose free coefficients are chosen to minimize relative weight variance using one score JVP per reference model.

Key results

• On text-to-image with SD v1.4, the method achieves the largest overall CLIP margin (0.1666) versus 0.1611 for DAS and 0.1465 for CFG and DNG.
• On pMHC binder design over 64 binders, mean specificity margin improves to 0.1856 from 0.1786 for target-A-only BoltzGen, with median 0.1822 vs 0.1587.
• Proposal optimization avoids the severe visual collapse seen with direct SMC in the "medieval feast" example.

Surprising findings

• Fixed guidance has the highest fraction of positive pMHC margins (93.8%), higher than the proposed method (84.4%).

Limitations

• Assumes stochastic SDE inference and a local one-step approximation; not applicable as stated to deterministic solvers, EDM parameterizations, or finite-time flow maps.
• Introduces application-specific hyperparameters and extra compute from particle sampling with two reference models.
• Binder evaluation uses one peptide pair with predicted iPTM rather than experimental affinity; no large-scale text-to-image specificity benchmark.

Experimental setup

Datasets: 40-component Gaussian mixture synthetic dataset CIFAR-10 DNG five-prompt text-to-image suite

Benchmarks: 40-MoG synthetic CIFAR-10 class removal CIFAR-10 multiclass selection DNG text-to-image prompts pMHC binder specificity (NLVPMVATV vs NLVFMVATV)

Metrics: Wrong% KL to uniform FID Recall CLIP similarity margin iPTM specificity margin ESS

Baselines: Composition Safe Latent Diffusion Dynamic Negative Guidance Safe Denoiser CFG with negative prompting Feynman-Kac Corrector Feynman-Kac Steering DriftLite DAS BoltzGen

Builds on: Dynamic Negative Guidance Safe Denoiser Feynman-Kac Steering DAS DriftLite

Closest papers by meaning

• Multilevel and Sequential Monte Carlo for Training-Free Diffusion Guidance (NeurIPS 2026 · similarity 0.89)
• A General Framework for Inference-time Scaling and Steering of Diffusion Models (ICML 2025 · similarity 0.88)
• Tempered Guided Diffusion (NeurIPS 2026 · similarity 0.88)
• Test-time Alignment of Diffusion Models without Reward Over-optimization (ICLR 2025 · similarity 0.88)
• Asymptotically Exact Negative Guidance of Diffusion Models via Positive-Unlabeled Learning (NeurIPS 2026 · similarity 0.88)
• Variance Reduction for Expectations with Diffusion Teachers (NeurIPS 2026 · similarity 0.87)