Research map / Image generation / Fast diffusion sampling

Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Fast-dLLM++, a training-free parallel decoding rule for masked diffusion LLMs that selects commit sets using the full sorted confidence profile (Fréchet lower/upper bounds) instead of the weakest selected token, reporting higher throughput than Fast-dLLM threshold and factor rules on LLaDA-8B.

Problem

Parallel decoding in diffusion LLMs must decide which masked tokens can be committed together safely; Fast-dLLM's factor rule compresses the confidence profile to its weakest selected token, discarding information and limiting parallelism.

Approach

Replace the weakest-confidence selector with a Fréchet bound-based score G_n = L_n - U_n computed from the full sorted confidence profile; this reduces exactly to Fast-dLLM's factor rule for flat profiles and adds a nonnegative heterogeneity bonus.

Key results

• On LLaDA-8B at length 256, Fréchet decoding reaches 1.36x average throughput over threshold decoding across eight dataset-length settings with 29.2% NFE reduction and 0.48-point average accuracy change.
• On LLaDA-V MathVista, throughput rises from 28.2 to 32.9 tokens/s over Fast-dLLM (speedup 11.6x vs full-step), while MathVerse accuracy drops from 28.6% to 27.9%.
• On Dream-v0-Base-7B, Fréchet achieves 1.28x average throughput over threshold with 21.1% NFE reduction.

Surprising findings

• On MathVerse, Fast-dLLM++ lowers accuracy (28.6% to 27.9%) while giving only a small throughput gain (23.3 to 23.8 tokens/s).
• Matched-factor comparisons show accuracy sometimes drops relative to threshold (e.g., MBPP factor 21.4 vs threshold 27.4 at length 256), so throughput gains do not always preserve accuracy.

Limitations

• Certificate may overcommit when confidences are miscalibrated or selected tokens are strongly coupled.
• Analysis restricted to greedy decoding; not extended to temperature or top-k sampling.
• Accuracy is not uniformly preserved; some settings show accuracy drops relative to baselines and margin sensitivity is task-dependent.

Experimental setup

Datasets: GSM8K MATH HumanEval MBPP MathVista MathVerse

Benchmarks: GSM8K MATH HumanEval MBPP MathVista MathVerse

Metrics: accuracy tokens per second NFE tokens per NFE speedup

Baselines: Fast-dLLM threshold decoding Fast-dLLM factor decoding LLaDA full-step decoding

Builds on: Fast-dLLM LLaDA masked diffusion models

Closest papers by meaning

• PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models (NeurIPS 2026 · similarity 0.92)
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.92)
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs (NeurIPS 2026 · similarity 0.91)
• DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.91)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models (ICML 2026 · similarity 0.90)