Research map / Image generation / Fast diffusion sampling
Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-based Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SVDD, an inference-time method that uses soft value functions to reweight per-step samples from pre-trained continuous or discrete diffusion models, enabling reward optimization without fine-tuning or differentiable reward models.
Problem
Optimizing downstream non-differentiable reward functions while preserving naturalness of pre-trained diffusion models, without fine-tuning and without differentiable proxies, including for discrete diffusion models.
Approach
At each denoising step, draw M candidate samples from the pre-trained denoiser and select one via categorical sampling weighted by exp(soft value / alpha), where values are estimated either by Monte Carlo regression (SVDD-MC) or by evaluating the reward on the posterior-mean estimate of x0 (SVDD-PM, no training).
Key results
- SVDD-PM improves Enhancer 50% quantile to 5.353 vs 4.28 (SMC) and 1.807 (Best-of-N) and 5'UTR 50% quantile to 1.214 vs 0.912 (Best-of-N).
- On molecule docking parp1, SVDD-MC reaches 12.00 median vs 10.00 for Best-of-N and 6.90 for SMC.
- Image compressibility 50% quantile improves from -101.4 (pre-trained) to -51.1 with SVDD-PM, versus -71.2 for Best-of-N.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVDD-PM instead of DPS | Stable Diffusion v1.5 · Image compressibility 50% quantile | compressibility score: -60.1 ± 0.44 -> -51.1 ± 0.38 | Helped |
| Use SVDD-PM instead of Best-of-N | Stable Diffusion v1.5 · Image aesthetic 10% quantile | aesthetic score: 6.34 ± 0.004 -> 6.47 ± 0.004 | Helped |
| Use SVDD-MC instead of Best-of-N | GDSS · Molecule SA 50% quantile | SA score: 0.834 ± 0.014 -> 0.935 ± 0.010 | Helped |
| Use SVDD-PM instead of Best-of-N | discrete diffusion model · Enhancers 10% quantile | activity: 3.449 ± 0.128 -> 6.980 ± 0.138 | Helped |
| Use SVDD-PM instead of Best-of-N | discrete diffusion model · 5'UTR 50% quantile | mean ribosomal load: 0.912 ± 0.023 -> 1.214 ± 0.016 | Helped |
| Increase duplication size M (ablation) | SVDD-PM · all domains | reward score | Mixed |
| Increase duplication size M (ablation) | SVDD-PM · all domains | computational complexity | Hurt |
| Increase duplication size M (ablation) | SVDD-PM · all domains | memory complexity | No clear effect |
| Use SVDD-MC rather than SVDD-PM | GDSS · Molecule QED 50% quantile | QED: 0.848 ± 0.014 -> 0.852 ± 0.011 | Helped |
| Use SVDD-PM rather than SVDD-MC | GDSS · Molecule QED 10% quantile | QED: 0.925 ± 0.007 -> 0.928 ± 0.008 | Helped |
Surprising findings
- Standard SMC reward maximization loses sample diversity even when reward is high, since effective sample size does not reflect real diversity.
- On molecule QED and SA, the simpler SVDD-PM is competitive with the learned-value SVDD-MC, and sometimes better (e.g., Enhancers, 5'UTR).
Limitations
- Inference cost roughly M times standard sampling (or M times memory if parallelized).
- Does not substantially move far from pre-trained model; RL fine-tuning may be stronger when large changes are needed.
- SVDD-PM applies reward directly to posterior-mean estimates, and with learned reward models over-optimization is not addressed.
Experimental setup
Datasets: ZINC-250k enhancer MPRA dataset 5'UTR MPRA dataset LAION aesthetic predictor training data
Benchmarks: Stable Diffusion compressibility and aesthetic prompts GDSS molecule QED, SA, docking (parp1, 5ht1b, jak2, braf) enhancer HepG2 activity 5'UTR mean ribosome load
Metrics: top 10% and 50% quantile reward validity uniqueness novelty FCD SNN NSPDK MMD molecule and atom stability
Baselines: Pre-trained Best-of-N DPS SMC (TDS-style) DG DiGress
Builds on: classifier guidance DPS soft Q-learning twisted SMC TDS
Closest papers by meaning
- Simple Guidance Mechanisms for Discrete Diffusion Models (ICLR 2025 · similarity 0.92)
- Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design (ICML 2025 · similarity 0.92)
- Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design (ICLR 2025 · similarity 0.91)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.91)
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.91)
- Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.