Research map / Image generation / Fast diffusion sampling

Accelerating Diffusion Language Models via Structured Suffix Modeling

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SSM, a training-free suffix-pruning method for diffusion language models that keeps different suffix token budgets in local, middle, and tail regions and mixes previous-step soft decoded embeddings into [MASK] inputs, yielding faster inference with competitive accuracy.

Problem

Diffusion language models must recompute attention and hidden states for all masked suffix tokens at every block-wise denoising step, which makes suffix computation a major latency bottleneck, especially for long generations.

Approach

Instead of uniformly dropping suffix tokens with a local window, partition the suffix into local, middle, and tail regions with region-specific retention (all local tokens, block start tokens in the middle, start and end in the tail), and carry information across steps by mixing top-k soft decoded embeddings of the previous step into the retained [MASK] embeddings.

Key results

• On LLaDA-Instruct HumanEval, SSM+Par. reaches 10.47x latency speedup over Vanilla (3.35s vs 35.09s) with pass@1 48.17 vs 43.90.
• In long-sequence GSM8K (1024 tokens, LLaDA-1.5), combining SSM with parallel decoding and prefix caching gives an overall 72.81x speedup over Vanilla.
• SSM+Par. on MBPP with LLaDA-Instruct achieves 23.44x latency speedup, about 2.3x over Streaming+Par.

Surprising findings

• Suffix pruning sometimes improves accuracy, e.g., strict-match on GSM8K rises from 37.38 (Vanilla) to 76.95 with SSM+Par., attributed to shorter, better-formatted outputs.
• Latency and TPS are not aligned; shorter generations can lower TPS while latency improves.

Limitations

• Efficiency gain depends on suffix length; limited when suffix is a small proportion of the sequence.
• May degrade accuracy on some datasets.
• Not evaluated on models larger than 10B parameters.

Experimental setup

Datasets: GSM8K MATH HumanEval MBPP

Benchmarks: GSM8K MATH HumanEval MBPP

Metrics: latency TPS generation length ratio flexible-extract accuracy strict-match accuracy pass@1

Baselines: Vanilla decoding Parallel decoding DPad Streaming-dLLM Margin-based parallel decoding

Builds on: DPad Streaming-dLLM Fast-dLLM

Closest papers by meaning

• Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference (NeurIPS 2026 · similarity 0.91)
• Encoder-Decoder Diffusion Language Models for Efficient Training and Inference (NeurIPS 2025 · similarity 0.91)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.90)
• dKV-Cache: The Cache for Diffusion Language Models (NeurIPS 2025 · similarity 0.90)
• Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.90)
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.90)