Research map / Image generation / Fast diffusion sampling

Mean-Field Parallel Decoding for Discrete Diffusion Language Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mean-Field Parallel Decoding, a training-free single-pass method that selects parallel commit sets in discrete diffusion LMs via a pairwise JSD-based MRF and mean-field fixed-point updates, improving throughput over entropy, KLASS, LocalLeap and DAWN baselines.

Problem

Parallel token commitment in discrete diffusion language models selects tokens by marginal confidence, causing joint inconsistency when several mutually dependent positions are unmasked simultaneously, which limits effective parallelism.

Approach

Treat the commit set as binary variables in a pairwise MRF whose unary term is top-2 log-margin and whose pairwise term is a JSD-based similarity between masked positions' predictive distributions; a mean-field fixed-point iteration (R=2) yields commit intensities that suppress conflicting simultaneous commitments without training.

Key results

• On LLaDA-8B-Instruct, the method reaches 4.93x speedup on GSM8K (78.75 TPS, 76.34 accuracy vs 79.15 for Entropy) and 6.10x on MBPP.
• On Dream-v0-Instruct-7B, MATH accuracy 40.44 vs 38.30 for Entropy at 3.61x speedup.
• Ablation on Dream-7B GSM8K: removing pairwise interaction gives 72.40 accuracy at 56.20 TPS, while R=1 drops accuracy to 54.51.

Surprising findings

• The R=1 one-shot variant performs much worse (54.51 accuracy on GSM8K) than the full iterated method, indicating a single mean-field step is insufficient.
• Removing pairwise interactions yields higher TPS than the full method on GSM8K (56.20 vs 49.13), so the interaction term trades throughput for accuracy.

Limitations

• Pairwise JSD cost O(m^2|V|) may need approximation for larger blocks.
• JSD interaction is a lightweight proxy and does not model higher-order structure or the true joint distribution.
• Best quality-latency configuration (block size, threshold tau) varies across models and tasks.

Experimental setup

Datasets: GSM8K MATH HumanEval MBPP

Benchmarks: GSM8K MATH HumanEval MBPP

Metrics: accuracy tokens per second speedup NFE

Baselines: Entropy decoding KLASS LocalLeap DAWN

Builds on: DAWN KLASS LocalLeap Markov Random Field mean-field variational inference

Closest papers by meaning

• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.92)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.92)
• Latent Refinement Decoding: Enhancing Diffusion Language Models by Refining Belief States (NeurIPS 2026 · similarity 0.90)
• DMax: Aggressive Parallel Decoding for dLLMs (NeurIPS 2026 · similarity 0.90)
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion (NeurIPS 2026 · similarity 0.90)
• Reconsidering Positional Supervision in Masked Diffusion Language Model Training (NeurIPS 2026 · similarity 0.90)