Research map / Image generation / Fast diffusion sampling

Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FreeDave, a training-free, model-free draft-and-verify decoding for diffusion LLMs that reuses the model's own forward passes to verify parallel-decoded drafts, giving lossless acceleration of up to 2.83x.

Problem

Parallel decoding in diffusion LLMs speeds up inference but threshold-based heuristics degrade quality relative to one-token-per-step static decoding, especially with large block sizes.

Approach

Each forward pass of the DLLM both verifies drafts produced at the previous step and supplies distributions for generating new multi-step drafts, so draft generation and verification are byproducts of static decoding; a verifier h(n,d) finds the largest feasible step merge that exactly reproduces the static oracle path.

Key results

• FreeDave on TraDo-8B-Instruct (block size 4) keeps GSM8K accuracy at 92.95 vs 93.03 static while raising TPS 1.84x and TPF 2.31x.
• Threshold-based parallel decoding on Dream-7B-Instruct with block size 32 drops MATH500 accuracy from 37.40 to 0.00, while FreeDave keeps 36.60 with 2.76x TPF.
• Peak TPF of 2.83x on HumanEval with Dream-7B-Instruct; TPS up to 2.36x across settings.

Surprising findings

• Threshold-based parallel decoding with Dream-7B-Instruct at block size 32 collapses to near-zero accuracy on math and code due to repeated tokens from overconfidence.
• Observed accuracy differences between static and FreeDave decoding are attributed mainly to GPU non-batch-invariant kernels and numerical non-determinism rather than the algorithm.

Limitations

• Speedup in TPS is smaller than TPF gains due to larger batch forward cost and memory overhead.
• Evaluated only offline with single-prompt inference on a single GPU; online serving not studied.
• Accuracy is not always bit-identical to static decoding because of GPU kernel non-determinism.

Experimental setup

Datasets: MATH500 GSM8K MBPP HumanEval AIME2025

Benchmarks: MATH500 GSM8K MBPP HumanEval AIME2025

Metrics: accuracy TPS TPF NFE response time

Baselines: static decoding threshold-based parallel decoding

Builds on: speculative decoding Fast-dLLM block diffusion static decoding

Closest papers by meaning

• DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.92)
• Mean-Field Parallel Decoding for Discrete Diffusion Language Models (NeurIPS 2026 · similarity 0.92)
• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference (NeurIPS 2026 · similarity 0.92)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.91)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs (NeurIPS 2026 · similarity 0.90)