Research map / Image generation / Fast diffusion sampling

EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Masked diffusion language model decoding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EntropyCache, a training-free KV caching method for diffusion LLMs that triggers full recomputation when the max entropy of newly decoded tokens exceeds a threshold and recomputes the k most recently decoded tokens otherwise.

Problem

Approximate KV caching for bidirectional diffusion LLMs is needed to cut the cost of full forward passes at every denoising step, but existing dynamic methods have decision overheads that scale with context length or model depth.

Approach

Uses the maximum entropy of newly decoded token distributions as an O(V) scalar signal for when to do a full recompute (a proxy for KV drift), and recomputes a fixed budget of k most recently decoded tokens since the last full pass, since feature volatility persists for multiple steps after unmasking.

Key results

• On standard benchmarks, EntropyCache reaches 52.28% average accuracy at 15.2x speedup on LLaDA-8B-Instruct and 56.21% at 26.4x on Dream-7B-Instruct.
• On chain-of-thought benchmarks, average speedup is 22.4x (LLaDA) and 24.1x (Dream) relative to baseline, about 1.4x the next-best method (d2Cache).
• Decision overhead is 0.5% of inference time versus 9.2% for d2Cache and 14.5% for Elastic-Cache on a single GSM8K sample.

Surprising findings

• Lowest accuracy drop on HumanEval despite the weakest entropy-drift correlation there (Spearman 0.377), suggesting entropy captures generation-critical steps rather than pure feature drift.
• MBPP accuracy on LLaDA falls 3.4 points below baseline at default settings.

Limitations

• MBPP accuracy on LLaDA drops below baseline (38.80 vs 42.20) at default parameters.
• Evaluated only on two dLLM architectures (LLaDA-8B and Dream-7B).
• Requires tuning of threshold tau and budget k per workload for best accuracy-throughput tradeoff.

Experimental setup

Datasets: GSM8K MATH500 MBPP HumanEval MMLU-Pro BBH GPQA

Benchmarks: GSM8K MATH500 MBPP HumanEval MMLU-Pro BBH GPQA

Metrics: accuracy throughput (tok/s) speedup recompute ratio decision overhead fraction

Baselines: Baseline+Parallel Fast-dLLM (Dual) Elastic-Cache d2Cache

Builds on: Fast-dLLM d2Cache Elastic-Cache LLaDA Dream

Closest papers by meaning

• dKV-Cache: The Cache for Diffusion Language Models (NeurIPS 2025 · similarity 0.94)
• Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs (NeurIPS 2026 · similarity 0.91)
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.91)
• KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.89)
• CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences (ICLR 2025 · similarity 0.89)
• SCBench: A KV Cache-Centric Analysis of Long-Context Methods (ICLR 2025 · similarity 0.89)