Research map / Large language models / Reasoning and chain of thought
R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes R-KV, a training-free decoding-time KV cache compression method for reasoning models that jointly scores token importance (attention) and redundancy (key-vector cosine similarity) to retain near-full accuracy with 10-34% of the cache.
Problem
Reasoning models generate very long, repetitive chain-of-thought outputs, causing huge KV caches during decoding; existing attention-based KV compression methods, designed for prefilling, retain redundant tokens and fail on reasoning tasks.
Approach
Combine attention-based importance with a redundancy score from semantic (cosine) similarity of key vectors, selecting tokens via a joint score Z = λI − (1−λ)R, applied periodically during decoding with a fixed buffer.
Key results
- R-KV on R1-Llama-8B reaches lossless compression with 34% KV budget on MATH-500 and 10% on AIME24, and reaches 105% of FullKV accuracy with 16% budget on AIME24.
- At 16K generation length, R-KV at 10% ratio gives 6.6x throughput over FullKV on Llama3-8B-based setup (up to 9x batch size).
- SnapKV at comparable budgets reaches only about 60% of full performance per the abstract, while R-KV reaches near 100% at 10% budget.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use R-KV instead of SnapKV with budget 1,024 | Llama3-8B · MATH | accuracy: 74.43 -> 81.34 | Helped |
| use R-KV instead of SnapKV with budget 1,024 | Llama3-8B · AIME24 | accuracy: 15.73 -> 45.26 | Helped |
| use R-KV instead of SnapKV with budget 1,024 | Qwen-14B · MATH | accuracy: 86.63 -> 90.72 | Helped |
| use R-KV instead of SnapKV with budget 1,024 | Qwen-14B · AIME24 | accuracy: 25.00 -> 42.66 | Helped |
| use R-KV with budget 1,024 instead of FullKV | Llama3-8B · MATH | accuracy: 82.38 -> 81.34 | Hurt |
| use R-KV with budget 1,024 instead of FullKV | Llama3-8B · AIME24 | accuracy: 49.79 -> 45.26 | Hurt |
| use R-KV with budget 1,024 instead of FullKV | Qwen-14B · MATH | accuracy: 94.58 -> 90.72 | Hurt |
| use R-KV with budget 1,024 instead of FullKV | Qwen-14B · AIME24 | accuracy: 65.68 -> 42.66 | Hurt |
| use redundancy-only selection (λ=0) instead of balanced importance and redundancy (ablation) | R-KV on DeepSeek-R1-Distill-Llama-8B · MATH-500 | accuracy | Hurt |
| use attention-only selection (λ=1) instead of balanced importance and redundancy (ablation) | R-KV on DeepSeek-R1-Distill-Llama-8B · MATH-500 | accuracy | Hurt |
| replace max pooling with mean pooling to aggregate GQA attention scores (ablation) | R-KV · MATH reasoning evaluation | accuracy | Hurt |
Surprising findings
- Repetitive content receives disproportionately high attention scores, so attention-only eviction (SnapKV) retains redundant tokens and fails on reasoning.
- R-KV exceeds FullKV accuracy (105%) at 16% budget on AIME24 for R1-Llama-8B.
Limitations
- Compatibility with paged attention and advanced attention mechanisms is non-trivial.
- Benefits may be reduced in serving frameworks lacking native KV compression interfaces due to reallocation overhead.
- Evaluated only on math reasoning benchmarks (MATH-500, AIME 2024) and two DeepSeek-R1 distilled models.
Experimental setup
Datasets: MATH-500 AIME 2024
Benchmarks: MATH-500 AIME 2024
Metrics: pass@1 (64 samples) accuracy ratio to FullKV memory saving throughput batch size
Baselines: SnapKV FullKV
Builds on: SnapKV PyramidKV H2O
Closest papers by meaning
- BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference (ICML 2026 · similarity 0.92)
- Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression (ICML 2026 · similarity 0.92)
- Value-Aware Stochastic KV Cache Eviction for Reasoning Models (NeurIPS 2026 · similarity 0.92)
- ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models (ICLR 2026 · similarity 0.92)
- Which Heads Matter for Reasoning? RL-Guided KV Cache Compression (ICML 2026 · similarity 0.91)
- Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.