Research map / Image generation / Fast diffusion sampling

Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction

ICLR 2025 · Image generation › Fast diffusion sampling · approach: Discrete diffusion solvers · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Discrete Denoising Posterior Prediction (DDPP), a simulation-free framework that steers pretrained masked diffusion models toward reward-induced Bayesian posteriors via three objectives (IS, LB, KL), validated on grids, MNIST, CelebA, protein language models, and text.

Problem

Steering pretrained masked discrete diffusion models toward non-differentiable or differentiable rewards without ground-truth posterior samples, in a scalable way; existing RLHF-style fine-tuning of diffusion models optimizes loose bounds and discrete guidance lacks principled definitions.

Approach

Cast MDM steering as sampling from a reward-tilted posterior proportional to the pretrained MDM times a reward, and fine-tune a second MDM to match the denoising reward-induced posterior using a simulation-free log-ratio regression; three variants estimate the log partition function by importance sampling (DDPP-IS), a learned lower bound (DDPP-LB), or a Reinmax reverse-KL gradient that avoids it (

Key results

• On even-digit MNIST, DDPP-KL reaches FLD 31.75 vs base 33.89 and RTB 45.97, with log R -3.13 vs RTB -18.66.
• On protein beta-sheet task, DDPP-LB reaches beta-sheet 0.436 and pLDDT 0.897, outperforming RTB (0.319) and Best-of-10 (0.280).
• On Tinystories, DDPP-LB reaches log reward 205.76 versus 107.83 for RTB and 146.95 for SVDD, with Gen PPL 19.60.

Measured effects reported in the paper

Surprising findings

• On MNIST, SVDD achieves higher average log R than DDPP but is worse on sample-based FLD and class-conditional BPD.
• On CelebA, DDPP-LB obtains BPD within the base model range but worse than RTB, despite the large reward gain.

Limitations

• DDPP-KL requires a differentiable reward, limiting applicability.
• DDPP-LB requires a learned partition-function head and warm-up to stabilize training.
• Wet-lab expression yields were lower than positive controls and many designs were insoluble.

Experimental setup

Datasets: CelebA binarized MNIST Tinystories Amazon reviews

Benchmarks: binarized MNIST even-digit steering CelebA 64x64 blond-hair steering protein beta-sheet and shrinking in-silico tasks Tinystories toxic story generation Amazon review generation

Metrics: log reward FLD BPD pLDDT pTM beta-sheet % TM-Score SS-KL Gen PPL SDS-PAGE expression

Baselines: Best-of-N RTB SVDD base model

Builds on: masked diffusion models relative trajectory balance Reinmax DPLM

Closest papers by meaning

• Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design (ICLR 2025 · similarity 0.91)
• Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-based Decoding (NeurIPS 2025 · similarity 0.91)
• Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.89)
• Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design (ICML 2025 · similarity 0.89)
• Diffusion Language Models Are Versatile Protein Learners (ICML 2024 · similarity 0.89)
• D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform (NeurIPS 2026 · similarity 0.89)