Research map / Large language models / Language, safety and interpretability
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adaptive parallel decoding (APD) accelerates diffusion LLMs by sampling variable-length token groups in parallel, accepting tokens according to a product-of-experts mixture of the dLLM marginals and a small autoregressive model's joint probability.
Problem
Diffusion LLMs promise parallel token generation but in practice lose substantial quality when decoding many tokens per step, and remain far slower than autoregressive models at comparable quality.
Approach
Fix the dLLM to left-to-right order so it behaves autoregressively, then use a small AR model (Qwen2.5 0.5B) to verify parallel proposals via a multiplicative mixture p_T ∝ p_D^R * p_AR^(1-R) with universal Gumbel coupling, accepting a variable number of tokens per iteration; add KV caching outside a sliding window and a maximum masked lookahead.
Key results
- Dream 7B with APD (R=0.7, W=16, M=100) averages 7.62 parallel tokens per iteration on GSM8K while staying close to the one-token-per-step accuracy.
- On GSM8K, Dream 7B left-to-right (256 steps) reaches 0.832 accuracy at 10.1 tokens/sec, while Qwen2.5 7B autoregressive reaches 0.854 at 38.6 tokens/sec, showing dLLMs lag AR speed.
- Dream with APD is reported to be faster than autoregressive Qwen2.5 7B and even Qwen2.5 0.5B on the speed-quality Pareto frontier with minimal quality loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use left-to-right decoding instead of entropy-based decoding (ablation) | Dream 7B · GSM8K | accuracy: 0.804 -> 0.832 | Helped |
| Use left-to-right decoding instead of confidence-based decoding (ablation) | Llada 8B · GSM8K | accuracy: 0.534 -> 0.712 | Helped |
| Reduce decoding steps from 256 to 128 (ablation) | Dream 7B with entropy decoding · GSM8K | accuracy: 0.804 -> 0.708 | Hurt |
| Reduce decoding steps from 256 to 128 (ablation) | Llada 8B with confidence decoding · GSM8K | accuracy: 0.534 -> 0.526 | No clear effect |
| Increase parallel tokens sampled per iteration (ablation) | Diffusion language models · GSM8K | accuracy | Hurt |
| Increase multiplicative mixture weight R (ablation) | APD with Dream 7B · GSM8K | accuracy | Hurt |
| Decrease recompute KV window W (ablation) | Left-to-right Dream 7B decoding · GSM8K | throughput | Helped |
| Decrease maximum masked lookahead M (ablation) | Dream 7B decoding · GSM8K | throughput | Helped |
| Use limited masked input with lower maximum lookahead M (ablation) | Dream 7B on complex reasoning tasks · MATH | quality | Hurt |
| Use APD instead of fixed parallel sampling (ablation) | Dream 7B · GSM8K | accuracy | Helped |
| Enable KV caching with a sliding recompute window (ablation) | Dream 7B diffusion decoding · GSM8K | throughput | Helped |
Surprising findings
- Left-to-right decoding of Dream 7B gives better GSM8K accuracy than the default entropy-based unmasking order.
- Reducing the diffusion model's recompute KV window and masked lookahead gives speedups with little quality loss, though shorter masked lookahead can shorten generations and hurt reasoning accuracy.
Limitations
- Higher throughput comes with some quality degradation (a tradeoff, not free).
- APD cannot exceed the quality of the base diffusion model it samples from; weak domains of Dream carry over.
- Parallel acceptance rate is lower on open-ended or less regular text than on math reasoning.
Experimental setup
Datasets: GSM8K GPQA MATH HumanEval persuasive writing prompts
Benchmarks: GSM8K GPQA MATH HumanEval
Metrics: accuracy throughput (tokens/sec) average parallel tokens per iteration
Baselines: Dream 7B (K=1) Qwen2.5 7B Qwen2.5 0.5B Llada 8B Dream 7B with fixed parallel decoding
Builds on: speculative decoding Dream 7B Gumbel-Softmax trick KV caching block diffusion
Closest papers by meaning
- SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.93)
- AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism (ICML 2025 · similarity 0.92)
- Mean-Field Parallel Decoding for Discrete Diffusion Language Models (NeurIPS 2026 · similarity 0.92)
- DFlash: Block Diffusion for Flash Speculative Decoding (ICML 2026 · similarity 0.91)
- A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.91)
- Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.