Research map / Image generation / Fast diffusion sampling

DMax: Aggressive Parallel Decoding for dLLMs

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DMax, which trains a masked diffusion LLM with on-policy uniform training and decodes with soft parallel decoding (mask/token embedding interpolation), greatly raising tokens-per-forward on math and code benchmarks while keeping accuracy.

Problem

Parallel decoding in masked diffusion language models degrades sharply because early errors are committed and propagate (error accumulation), limiting practical speedup.

Approach

Extend a pretrained MDLM into a self-correcting uniform model by training on noisy inputs sampled on-policy from the model's own predictions (OPUT), and decode with soft embeddings that interpolate the top-1 token embedding and the mask embedding by confidence (SPD), allowing iterative self-revision in embedding space.

Key results

• On GSM8K, TPF rises from 2.04 (LLaDA-2.0-mini) to 5.48 (DMax-Math) with accuracy 92.1% vs 92.6%.
• On MBPP, TPF rises from 2.71 to 5.86 with accuracy 79.2% vs 80.6% for DMax-Coder.
• On two H200 GPUs, the model reaches about 1,338 TPS at batch size 1 (per abstract).

Surprising findings

• Applying soft parallel decoding to a standard masked diffusion model without OPUT causes catastrophic collapse (GSM8K accuracy near 0%).

Limitations

• Soft parallel decoding requires OPUT-trained models and fails on standard MDLMs.
• Accuracy drops slightly on some benchmarks (e.g., MBPP 80.6% to 79.2%, MATH500 75.8% to 75.4%) at high parallelism.

Experimental setup

Datasets: GSM8K trainset PRM12K Numina-Math OpenThoughts OpenCodeInstruct

Benchmarks: GSM8K MATH500 Minerva-Algebra ASDIV HumanEval-Instruct MBPP-Instruct

Metrics: TPF TPS accuracy AUP score

Baselines: LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training

Builds on: LLaDA-2.0 masked diffusion language models uniform diffusion language models dParallel dInFer

Closest papers by meaning

• Mean-Field Parallel Decoding for Discrete Diffusion Language Models (NeurIPS 2026 · similarity 0.90)
• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference (NeurIPS 2026 · similarity 0.90)
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.89)
• AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism (ICML 2025 · similarity 0.89)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.88)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.88)