Research map / Large language models / Reasoning and chain of thought

DFlash: Block Diffusion for Flash Speculative Decoding

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Speculative decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DFlash, a speculative decoding framework where a lightweight block-diffusion drafter, conditioned on injected target-model hidden features via KV injection, drafts a whole token block in one forward pass for parallel verification.

Problem

Autoregressive drafting in speculative decoding is sequential and caps speedups at roughly 2-3x, while existing diffusion drafters are either too large (7B) or too weak to give high acceptance lengths.

Approach

Use a small block-diffusion drafter that is conditioned on fused hidden features from the frozen target LLM, injected as Key/Value entries into every draft layer, so that deeper drafters stay cheap (one parallel pass) and keep high acceptance length; training uses random anchor-token block sampling and exponentially decaying position loss.

Key results

• On Qwen3-8B with thinking disabled at temperature 0, DFlash (block 16) reaches about 4.9x average speedup over autoregressive decoding, roughly 2.4x over EAGLE-3 (tree 16), with peak 6.1x on MATH-500.
• On SGLang with a single B200, DFlash achieves up to 5.1x speedup on Qwen3-8B at concurrency 1 (Math500).
• Without target features, a 5-layer block diffusion drafter reaches only about 2-3x speedup, showing the target context is essential.

Surprising findings

• A block-diffusion drafter without target-model features gives only modest speedups (about 2-3x), indicating conditioning is critical.
• Models trained with a larger block size generalize to smaller inference-time block sizes, but not the reverse.

Limitations

• Speedup declines at high concurrency (e.g., 2.3-2.9x at concurrency 32 on SGLang), since large blocks raise verification cost in compute-bound settings.
• Comparisons with other diffusion-based speculative decoding methods (DiffuSpec, SpecDiff-2, TiDAR) were not possible due to lack of open-source implementations.
• Training requires caching or computing target hidden states, and offline storage grows linearly with the number of extracted features.

Experimental setup

Datasets: Nemotron Post-Training Dataset V2 CodeAlpaca UltraChat ShareGPT LongAlign-10K

Benchmarks: GSM8K MATH-500 AIME25 HumanEval MBPP LiveCodeBench MT-Bench Alpaca GPQA LongBench

Metrics: decoding speedup average acceptance length (tau) throughput (tok/s)

Baselines: EAGLE-3 autoregressive decoding PARD DiffuSpec SpecDiff-2 MTP

Builds on: EAGLE-3 block diffusion speculative decoding Flex Attention

Closest papers by meaning

• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.93)
• SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.92)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.91)
• BASTION: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting (NeurIPS 2026 · similarity 0.91)
• A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.91)
• Speculative Speculative Decoding (ICLR 2026 · similarity 0.91)