Research map / Image generation / Fast diffusion sampling

Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DyStruct, a training-free Bayesian structured decoding framework for diffusion language models that dynamically sizes new windows and partitions them into CRP-style blocks with context-aware scheduling and edge-welding.

Problem

Diffusion language models usually need a fixed generation length; existing flexible-length methods either require retraining or rely on local confidence heuristics that ignore evolving sequence structure.

Approach

Treats flexible-length diffusion decoding as joint Bayesian inference over window length, block partition (CRP prior with edge likelihood from instability and JSD features), and block decoding schedule (Gibbs distribution favoring stable, context-anchored blocks), plus boundary repair via localized remasking, all at inference time on a frozen model.</important>

Key results

• On LLaDA-8B-Base, DyStruct raises BBH exact match from 44.9 to 49.3 and MBPP from 39.8 to 41.4, outperforming DAEDAL on all five benchmarks.
• On Dream-7B-Base, DyStruct reaches 47.0 on HumanEval versus 40.2 for the base model, while DAEDAL drops to 34.7.
• Forcing a 256-token initial window collapses BBH accuracy from 49.3 to 46.3 in the paper's ablation.

Surprising findings

• DAEDAL degrades BBH and HumanEval performance on Dream-7B-Base relative to the unmodified base model.
• Forcing the full 256-token window induces severe distributional drift, collapsing BBH accuracy.

Limitations

• Purely inference-time method; structural inference is not integrated into training.
• Evaluated only on two 7-8B base DLMs and five benchmarks with a 256-token budget.
• Code subset in the significance test is small (664 instances), so the code gain is not statistically significant.

Experimental setup

Datasets: GSM8K MATH MBPP HumanEval BBH

Benchmarks: HumanEval MBPP GSM8K MATH BBH

Metrics: pass@1 exact match strict-match accuracy McNemar test tokens generated blocks

Baselines: LLaDA-8B-Base DAEDAL Dream-7B-Base

Builds on: DAEDAL Chinese Restaurant Process LLaDA Dream

Closest papers by meaning

• Accelerating Diffusion Language Models via Structured Suffix Modeling (NeurIPS 2026 · similarity 0.91)
• PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Confidence-Based Decoding is Provably Efficient for Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.90)
• DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.90)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.90)