Research map / Image generation / Fast diffusion sampling

Primal-Dual Guided Decoding for Constrained Discrete Diffusion

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes primal-dual guided decoding (SPDD), an inference-time method that adds Lagrangian-derived logit biases with mirror-descent multiplier updates to masked discrete diffusion (MDLM) to satisfy global linear constraints on text, molecules, and playlists.

Problem

Enforcing global property constraints (e.g., topical vocabulary counts, molecular weight, genre composition) during masked discrete diffusion generation while staying close to the model's unconstrained distribution, without retraining or costly inner loops.

Approach

Formulates constrained unmasking as a KL-regularised optimisation; the dual solution yields an additive, constraint-dependent logit bias, with the Lagrange multiplier updated online by mirror descent on accumulated constraint slack.

Key results

• On TinyStories ocean steering, SPDD reaches 92.1% Pass% vs 38.0% for CDD and 81.9% for static bias, with KL 0.420.
• On playlist generation, SPDD matches CDD's satisfaction (9.28% vs 9.18% Pass%) at KL 0.355 vs 0.779, with 1.03x sampling cost vs CDD's 7.28x.
• On molecular MW>=350, SPDD reaches 48.5% Pass% vs 42.8% for D-CBG and 42.2% for CDD.

Surprising findings

• D-CBG achieves 0.0% Pass% on the sparse text constraint despite a trained classifier, attributed to overly smooth Taylor-approximated gradients.

Limitations

• Experiments restricted to absorbing-state diffusion, where early token commitments cannot be revised.
• Constraint bounds are probabilistic, so strict feasibility is not guaranteed.
• Non-decomposable constraints (e.g., QED) need better partial-sequence slack estimators.

Experimental setup

Datasets: TinyStories ZINC20

Benchmarks: TinyStories ocean steering ZINC20 molecular weight steering playlist subgenre completion

Metrics: Pass% KL PPL Dist-2 validity

Baselines: Static logit bias D-CBG CDD GPT-4.1-mini LLaDA-8B

Builds on: MDLM D3PM CDD D-CBG Optimistic Mirror Descent

Closest papers by meaning

• Constrained Discrete Diffusion (NeurIPS 2025 · similarity 0.92)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata (NeurIPS 2026 · similarity 0.91)
• Search-Augmented Masked Diffusion Models for Constrained Generation (NeurIPS 2026 · similarity 0.90)
• DINGO: Constrained Inference for Diffusion LLMs (NeurIPS 2025 · similarity 0.89)
• Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-based Decoding (NeurIPS 2025 · similarity 0.89)
• Encoder-Decoder Diffusion Language Models for Efficient Training and Inference (NeurIPS 2025 · similarity 0.88)