Research map / Architectures and efficiency / Attention and transformers

Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Polestar is a training-free dLLM inference framework that uses KL-based token representation drift to drive sparse KV-cache refreshes (Polestar-Cache) and drift-aware early token commitment (Polestar-Commit), improving the accuracy-throughput Pareto frontier.

Problem

Diffusion LLM inference is slowed by bidirectional attention, which prevents standard KV-cache reuse, and by static confidence thresholds for parallel decoding that trade off accuracy against parallelism.

Approach

Token representation drift (KL divergence of a token's attention distribution across consecutive steps) is used as a unified signal: to pick which cached KV positions (clustered, quantized hidden states) to refresh sparsely, and to detect sharp drift spikes that signal commit-ready tokens in the current and suffix blocks.

Key results

• On GSM8K (256 gen length) with LLaDA-8B-Instruct, Polestar reaches 78.33% accuracy, 3.67 TPF and 87.57 TPS (13.3x over baseline), versus 79.30% and 6.58 TPS for the baseline.
• Polestar improves accuracy by up to 10.73% over KV-caching baselines and achieves up to 20.5x speedup over baseline.
• On ParallelBench-easy, Polestar achieves 2.43 TPF and 82.41 TPS (7.8x) with 74.18% accuracy.

Surprising findings

• Lowering the static confidence threshold to 0.7 raises TPF to 3.71 but causes a large accuracy drop, showing confidence-only scaling is suboptimal.
• Polestar improves accuracy over the full-recomputation baseline on some settings (e.g., GSM8K-512, HumanEval-512) while also increasing throughput.

Limitations

• Gains are highest on GSM8K/HumanEval; on some settings (e.g., MATH with LLaDA-8B, MBPP-512 LLaDA) Polestar roughly matches or trails baselines in accuracy.
• Many hyperparameters (K, top-k, tau_upd, alpha, window sizes) are tuned on GSM8K and reported via ablations.

Experimental setup

Datasets: GSM8K MATH HumanEval MBPP ParallelBench

Benchmarks: GSM8K MATH HumanEval MBPP ParallelBench MathVista MathVerse

Metrics: accuracy tokens per forward (TPF) tokens per second (TPS) pass@1

Baselines: Baseline+Parallel Fast-dLLM Dynamic-dLLM Elastic-Cache d$^2$Cache EntropyCache AdaBlock-dLLM DAWN KLASS

Builds on: Fast-dLLM Elastic-Cache LLaDA KLASS DAWN

Closest papers by meaning

• dKV-Cache: The Cache for Diffusion Language Models (NeurIPS 2025 · similarity 0.92)
• Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference (NeurIPS 2026 · similarity 0.91)
• EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models (NeurIPS 2026 · similarity 0.91)
• Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion (NeurIPS 2026 · similarity 0.91)
• DAWN: Dependency-Aware Fast Inference for Diffusion LLMs (NeurIPS 2026 · similarity 0.90)
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models (NeurIPS 2026 · similarity 0.90)