Research map / Large language models / Reasoning and chain of thought

Value-Aware Stochastic KV Cache Eviction for Reasoning Models

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: KV cache compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VaSE, a training-free KV cache eviction method for reasoning models that reserves slots for large-magnitude value states and adds stochastic sampling/resampled Gaussian projections, closing much of the accuracy gap to selection-based sparse attention.

Problem

KV cache eviction for long-chain-of-thought reasoning models loses substantially more accuracy than selection-based sparse attention, which keeps the full cache but activates a sparse subset.

Approach

Evicting a small set of large-magnitude value states causes repetitive reasoning loops, so VaSE reserves budget slots for high-Range value vectors and fills the rest by stochastic sampling over attention scores (or resamples the CurDKV Gaussian projection each eviction step) to increase cache diversity.

Key results

• On Qwen3-4B at ~4x compression, VaSE-AttnV averages 59.09% vs. R-KV 54.69% and SeerAttention-R 58.81%.
• On Qwen3-14B, VaSE-AttnV averages 65.81% vs. R-KV 60.90% and SeerAttention-R 65.37%.
• VaSE-DKV improves over CurDKV by 7.7 points (4B) and 9.2 points (14B) on average accuracy.

Surprising findings

• Evicting the largest-Range value states drops GSM8K accuracy to 14.3%, far below random eviction, causing repetitive reasoning loops.
• SeerAttention-R underperforms several training-free eviction methods on LiveCodeBench (45.3% vs. ~62% for R-KV), suggesting poor domain generalization of learned gates.

Limitations

• Only evaluated on Qwen3 models because SeerAttention-R releases checkpoints only for Qwen models.
• Focused on decode-phase eviction; prefill-phase compression not evaluated.
• Small test sets (30 problems on AIME) yield overlapping confidence intervals.

Experimental setup

Datasets: GSM8K

Benchmarks: AIME25 AIME26 HMMT25 GPQA-Diamond MATH LiveCodeBench-v6

Metrics: pass@1 decode throughput peak GPU memory reconstruction error

Baselines: SnapKV R-KV CurDKV SeerAttention-R Full KV cache

Builds on: SnapKV R-KV CurDKV FlashAttention2

Closest papers by meaning

• ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution (ICML 2026 · similarity 0.93)
• R-KV: Redundancy-aware KV Cache Compression for Reasoning Models (NeurIPS 2025 · similarity 0.92)
• ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models (ICLR 2026 · similarity 0.91)
• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference (ICML 2026 · similarity 0.90)
• DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction (NeurIPS 2026 · similarity 0.90)
• ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning (ICML 2026 · similarity 0.89)