Research map / Large language models / Reasoning and chain of thought

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

ICML 2026 · Large language models › Reasoning and chain of thought · approach: KV cache compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BeaconKV, a training-free KV cache eviction method for large reasoning models that scores cached KV pairs using continually selected FPS "beacon" queries plus recent queries, with max-aggregation.

Problem

KV cache grows linearly during long chain-of-thought reasoning; existing eviction methods score KV importance using only recent queries, which miss distant context revisited later by Thought Revisiting Tokens.

Approach

Maintain a bounded set of geometrically diverse pre-RoPE beacon queries chosen online by Continual Farthest Point Sampling, re-rotate them to the current step to anticipate future global attention, combine with recent queries, and max-pool attention scores to decide which KV pairs to keep.

Key results

• On Qwen3-14B AIME24 with a 1024 max KV budget, BeaconKV gains up to 31.7 points over existing compression methods.
• On Qwen3-4B with a 2K budget, BeaconKV reduces peak memory from 77.0 GB to 13.3 GB (5.8x) and raises throughput from 82.3 to 356.4 tokens/s versus Full KV.
• At a 1K budget on LiveCodeBench, BeaconKV reaches 42.2% accuracy versus 29.9% for RPC and 30.9% for SnapKV on Qwen3-4B.

Surprising findings

• Max aggregation outperforms mean aggregation in most budget settings, which the authors attribute to sparse but high-value TRT signals being diluted by mean pooling.
• On R1-Distill-Llama-8B and Qwen3-4B for GPQA-Diamond, the Initial+Recent baseline is close to or slightly above BeaconKV in some cases.

Limitations

• Evaluation limited to long-horizon reasoning tasks and open-source LRMs; generalization to non-reasoning long-context tasks is unexplored.
• Several hyperparameters (number of beacon and recent queries, budget) are fixed and their sensitivity is not systematically studied.
• Throughput is slightly lower than RPC at matched budgets and decoding latency is marginally higher.

Experimental setup

Datasets: AIME24 MATH-500 LiveCodeBench GPQA-Diamond

Benchmarks: AIME24 MATH-500 GPQA-Diamond LiveCodeBench

Metrics: pass@1 accuracy throughput (tokens/s) decode latency peak GPU memory

Baselines: SnapKV RPC R-KV Initial+Recent K-Means Centroids Naive FPS

Builds on: SnapKV RPC R-KV

Closest papers by meaning

• R-KV: Redundancy-aware KV Cache Compression for Reasoning Models (NeurIPS 2025 · similarity 0.92)
• ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models (ICLR 2026 · similarity 0.92)
• Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression (ICML 2026 · similarity 0.92)
• Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning (NeurIPS 2025 · similarity 0.90)
• ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning (ICML 2026 · similarity 0.90)
• ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution (ICML 2026 · similarity 0.90)