Research map / Large language models / Reasoning and chain of thought

ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution

ICML 2026 · Large language models › Reasoning and chain of thought · approach: KV cache compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ForesightKV, a learned KV cache eviction scorer for long reasoning traces, trained first by supervised pairwise ranking on "Golden Eviction" labels from future attention and then by GRPO with a low-entropy-token loss reward.

Problem

KV cache growth during long reasoning traces causes memory and latency costs, while existing eviction methods (rule-based or one-shot trained) fail to capture dynamic, semantic-dependent KV importance and cause large loss increases on low-entropy tokens.

Approach

Label KV pairs with future-attention-based "Golden Eviction" traces, train a lightweight MLP scorer with pairwise ranking loss, then refine the scorer with GRPO over sampled eviction trajectories, using a reward on squared loss increases of low-entropy tokens, with the LLM frozen throughout.

Key results

• On AIME2024 with Qwen3-4B, ForesightKV at a 1K budget scores 54.5 vs. 44.8 for R-KV at 2K.
• Efficiency on Qwen3-4B (one A800): ForesightKV-1K achieves up to 9.79x throughput over full cache at 32K generation length.
• Reward ablation: the MSE-based reward reaches 54.5 on AIME24 and 42.3 on AIME25 versus 51.7 and 40.9 for the supervised-only model at 1K budget.

Surprising findings

• Naively minimizing overall language-modeling loss as the RL reward does not improve performance (50.6 vs 51.7 on AIME24), and focusing on high-entropy tokens causes significant degradation (49.6).

Limitations

• Training requires generating long reasoning traces and computing full attention matrices for Golden Eviction labels, adding training cost.
• Main experiments use only math benchmarks (AIME) for primary evaluation; training is on math data.
• Some comparisons (G-KV, DuoAttention, RPC) use different or partially reported settings, e.g., G-KV results taken from original paper.

Experimental setup

Datasets: STILL SCP-116K

Benchmarks: AIME2024 AIME2025 GPQA LiveCodeBench-V3 LongBench

Metrics: pass@1 (averaged over 32 runs) LM loss ratio throughput maximum concurrent batch size cosine similarity of attention outputs

Baselines: H2O SnapKV R-KV RPC G-KV DuoAttention

Builds on: GRPO H2O SnapKV R-KV

Closest papers by meaning

• Value-Aware Stochastic KV Cache Eviction for Reasoning Models (NeurIPS 2026 · similarity 0.93)
• Learning to Evict from Key-Value Cache (ICML 2026 · similarity 0.92)
• Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference (NeurIPS 2025 · similarity 0.90)
• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference (ICML 2026 · similarity 0.90)
• R-KV: Redundancy-aware KV Cache Compression for Reasoning Models (NeurIPS 2025 · similarity 0.90)
• DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction (NeurIPS 2026 · similarity 0.90)