Research map / Large language models / Language, safety and interpretability

DAWN: Dependency-Aware Fast Inference for Diffusion LLMs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DAWN, a training-free decoding method for diffusion LLMs that builds a sparse attention-based dependency graph to relax confidence thresholds for anchor-coupled positions and schedule non-conflicting low-confidence positions in parallel, yielding 1.8-8.06x speedups with near-preserved accuracy.

Problem

Parallel decoding in diffusion LLMs assumes positions are independent, but strongly coupled tokens unmasked together cause errors; existing confidence/entropy-based parallel strategies are overly conservative and leave efficiency unexploited.

Approach

Estimate inter-token dependencies from thresholded, sink-filtered attention maps each iteration; unmask confident anchor-induced positions at a relaxed threshold and greedily select a conflict-free set of lower-confidence positions, so parallelism grows without coupled errors.

Key results

• On LLaDA-1.5 MBPP, DAWN reaches 8.06x speedup (27.80 TPS) with accuracy 37.60 versus 38.80 for the original sampler.
• On LLaDA-8B-Instruct GSM8K, DAWN matches the original accuracy (77.94) at 4.33x speedup (44.72 TPS vs 10.32).
• Across the four models and four benchmarks, DAWN reports 1.80x to 8.06x speedups over the top-1 baseline while keeping accuracy close to the original.

Surprising findings

• Removing Conflict-Based Scheduling on Dream-v0-Instruct-7B GSM8K raised accuracy (75.28 vs 73.16) but reduced speed, indicating a trade-off.
• On LLaDA-8B-Instruct MBPP, DAWN slightly exceeds the original sampler accuracy (30.80 vs 29.60).

Limitations

• Hyperparameters (tau_edge, tau_induced, tau_sink, tau_low) are tuned per model and benchmark via grid search, including on HumanEval which is also a test benchmark.
• Accuracy is sometimes slightly below the original sampler (e.g., MATH and HumanEval for some models), so quality is not strictly preserved.
• Evaluated only on two model families (LLaDA and Dream) at generation length 256.

Experimental setup

Datasets: GSM8K MATH HumanEval MBPP

Benchmarks: GSM8K MATH HumanEval MBPP

Metrics: accuracy tokens per second speedup number of function evaluations

Baselines: Top-1 sampling (original) Fast-dLLM confidence-aware parallel KLASS LocalLeap

Builds on: Fast-dLLM LLaDA Dream KLASS LocalLeap

Closest papers by meaning

• Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding (ICLR 2026 · similarity 0.92)
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.92)
• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference (NeurIPS 2026 · similarity 0.91)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.90)
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs (NeurIPS 2026 · similarity 0.90)