Research map / Large language models / Language, safety and interpretability

PACE-dLLM: Elastic Block Decoding via Confidence Cliff Estimation for Diffusion Language Models

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Parallel and early-exit decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PACE-dLLM, which fits a logistic confidence cliff in closed form at each decoding step to set an elastic look-ahead horizon for block-wise diffusion LLM decoding, decoupled from a Fast-dLLM-style confidence commit threshold, reporting speedups of 5.23x on LLaDA and 3.06x on Dream.

Problem

Block-wise semi-autoregressive decoding of diffusion LLMs couples the look-ahead horizon with the number of tokens committed per step through a single block size B, forcing a quality-speed trade-off.

Approach

Per-position top-1 confidence inside the active window follows a context-dependent cliff; a closed-form logistic fit to it gives the largest distance at which expected confidence stays above an anchor, which sets the next horizon, while a separate confidence threshold decides commits. The cursor slides past resolved positions and the commit threshold is tightened at the horizon boundary.

Key results

• On LLaDA-8B-Instruct, PACE-dLLM averages 51.75 accuracy versus 48.41 for Fast-dLLM and 5.23x average speedup over the default semi-AR decoder.
• On Dream-7B-Instruct, PACE-dLLM averages 60.12 accuracy (best) with 3.06x average speedup, slower than Fast-dLLM's 3.53x.
• Up to 8.52x speedup on MATH-500 with LLaDA (109.9 tok/s vs 12.9 default).

Surprising findings

• On Dream, Fast-dLLM and AdaBlock-dLLM run faster than PACE-dLLM, which the authors attribute to Dream's flatter, partially non-monotone cliff.
• A single shared horizon rail [24,36] for all tasks gives 51.65 average on LLaDA, nearly matching the task-family rails.

Limitations

• Depends on checkpoint confidence calibration and shared controller defaults such as tau_commit and c_T.
• Oracle theory assumes a monotone cliff; the runtime logistic estimator lacks finite-sample guarantees and falls back to the upper rail on non-monotone profiles.
• Evaluated only on two semi-autoregressive 7B/8B backbones and L=512.

Experimental setup

Datasets: GSM8K MATH-500 HumanEval MBPP

Benchmarks: GSM8K MATH-500 HumanEval MBPP

Metrics: accuracy throughput (tok/s) wall-clock speedup NFE

Baselines: Default semi-AR decoder Fast-dLLM AdaBlock-dLLM DepCap

Builds on: Fast-dLLM LLaDA Dream low-confidence remasking

Closest papers by meaning

• Semantic-Aware Diffusion LLM Inference With Adaptive Block Size (ICLR 2026 · similarity 0.92)
• Confidence-Based Decoding is Provably Efficient for Diffusion Language Models (NeurIPS 2026 · similarity 0.92)
• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference (NeurIPS 2026 · similarity 0.92)
• Dynamically Structured Diffusion Language Model Decoding via Bayesian Inference (NeurIPS 2026 · similarity 0.91)
• SimSD: Simple Speculative Decoding in Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs (NeurIPS 2026 · similarity 0.90)