Research map / Large language models / Reasoning and chain of thought

Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: KV cache compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reasoning Path Compression (RPC), a training-free periodic KV cache compression method for reasoning LLMs that scores past tokens by attention from a recent selector window, achieving up to 1.60x throughput on QwQ-32B with 1.2% AIME 2024 accuracy drop.

Problem

Long reasoning paths from reasoning LLMs inflate KV cache memory and reduce token generation throughput, and existing KV cache compression methods target long prompts or fixed budgets rather than long generated reasoning traces.

Approach

Exploit semantic sparsity of reasoning traces: every P generated tokens, re-score all retained and new KV entries by attention from the most recent R selector queries (with local pooling across head and layer aggregation) and keep the top fraction at a fixed compression ratio, so compression is periodic rather than per-step and needs no training or budget set in advance.

Key results

• On QwQ-32B, RPC (P=4096) reaches 78.3% on AIME 2024 vs 79.5% full KV cache (1.2% drop), and H2O/TOVA reach 75.0% and 70.0%.
• RPC improves throughput by 1.68x at 32768 generated tokens for DeepSeek-R1-Distill-Qwen-7B and 1.60x at 16384 tokens for QwQ-32B.
• For DeepSeek-R1-Distill-Qwen-7B, RPC reduces peak memory at 32768 tokens from 75.7GB to 36.2GB; QwQ-32B at 32768 tokens runs out of memory with full KV cache but works with RPC.

Measured effects reported in the paper

Surprising findings

• LightThinker, a training-based reasoning path compression method, drops DeepSeek-R1-Distill-Qwen-7B to 6.7% on AIME 2024 despite only a mild compression ratio.
• Small selector windows (R=1 or 8) degrade accuracy noticeably, and very large R (128) also hurts, so the selector window must be tuned.

Limitations

• Accuracy drop on DeepSeek-R1-Distill-Qwen-7B is 2.6% at P=4096, larger than on QwQ-32B.
• Compression interval P and selector window R need tuning; accuracy improves with larger P at the cost of memory and throughput.
• Evaluation covers only two models and three benchmarks, with a fixed 4x compression ratio.

Experimental setup

Datasets: AIME 2024 LiveCodeBench IFEval HelloBench GSM8K GPQA-Diamond

Benchmarks: AIME 2024 LiveCodeBench IFEval

Metrics: pass@1 token generation throughput (tokens/s) peak memory (GB) redundancy rate 3-gram Shannon entropy

Baselines: Full KV Cache H2O TOVA LightThinker

Builds on: H2O TOVA SnapKV FlashAttention-2

Closest papers by meaning

• R-KV: Redundancy-aware KV Cache Compression for Reasoning Models (NeurIPS 2025 · similarity 0.91)
• When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models (ICLR 2026 · similarity 0.91)
• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference (ICML 2026 · similarity 0.90)
• TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs (NeurIPS 2025 · similarity 0.90)
• Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.90)
• Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression (ICML 2026 · similarity 0.90)