Research map / Architectures and efficiency / Attention and transformers
HARD-KV: Head-Adaptive Regularization for Decoding-time KV Compression
Yuxuan Yang, Feiyang Ren, Bowen Zeng, Dalin Zhang, Jinpeng Chen, Gang Chen, Huan Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add HARD-KV regularized Top-p budgeting | R-KV, Qwen3-8B · AIME24 (Top-k budget 4096) | accuracy: 43.3% -> 57.9% | Improved |
| add HARD-KV | R-KV, Qwen3-8B · AIME25 (Top-k budget 4096) | accuracy: 40.0% -> 56.7% | Improved |
| add HARD-KV | SnapKV, Qwen3-8B · AIME24 (Top-k budget 4096) | accuracy: 57.7% -> 56.7% | No clear effect |
| add HARD-KV | SnapKV, Qwen3-8B · AIME25 (Top-k budget 4096) | accuracy: 27.9% -> 42.3% | Improved |
| add HARD-KV | Vanilla, Qwen3-8B · U-Math (Top-k budget 1024) | accuracy: 9.8% -> 46.1% | Improved |
| add HARD-KV | Vanilla, Qwen3-8B · AIME25 (Top-k budget 2048) | accuracy: 16.7% -> 12.9% | Hurt |
| add HARD-KV with Top-p 0.90 dynamic budget | SnapKV, Qwen3-8B · AIME24 throughput | tokens/s: 301 -> 632 | Improved |
| add HARD-KV with Top-p 0.90 dynamic budget | Vanilla, Qwen3-8B · AIME24 throughput | tokens/s: 279 -> 732 | Improved |
| add HARD-KV with Top-p 0.90 dynamic budget | Vanilla, Qwen3-8B · AIME25 | accuracy: 33.3% -> 20.7% | Hurt |
| add CUDA Graph to base system (ablation) | inference system · AIME24 | throughput (tokens/s): 346 -> 477 | Improved |
| add Top-k pruning on top of CUDA Graph (ablation) | inference system · AIME24 | throughput (tokens/s): 477 -> 806 | Improved |
| increase Top-p budget from p70 to p90 (ablation) | SnapKV, Qwen3-8B · AIME25 | accuracy: 36.6% -> 42.3% | Improved |
Closest papers by meaning
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.89)
- Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction (NeurIPS 2026 · similarity 0.88)
- CompilerKV: Risk-Adaptive KV Cache Compression via Offline Experience Compilation (NeurIPS 2026 · similarity 0.88)
- RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression (ICML 2025 · similarity 0.88)
- Beyond Speedup - Utilizing KV Cache for Sampling and Reasoning (ICLR 2026 · similarity 0.88)
- SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
