Research map / Architectures and efficiency / Attention and transformers
HitKV: Activation Frequency Knows Which Tokens Are Important
Sanle Zhao, Yujuan Tan, Yu Jing, Zhuoxin Bai, Yue Niu, Jiayi Guo, Zongjie Wang, Ao Ren
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace PyramidKV eviction indicator with hit rate (P-HitKV) | PyramidKV Llama3-8B-Instruct · LongBench avg, B=128L | average score: 36.35 -> 37.30 | Improved |
| replace PyramidKV eviction indicator with hit rate (P-HitKV) | PyramidKV Llama3-8B-Instruct · LongBench avg, B=1024L | average score: 40.93 -> 40.97 | Improved |
| replace PyramidKV eviction indicator with hit rate (P-HitKV) | PyramidKV Mistral-7B-Instruct-v0.2 · LongBench avg, B=128L | average score: 35.67 -> 36.41 | Improved |
| replace PyramidKV eviction indicator with hit rate (P-HitKV) | PyramidKV Mistral-7B-Instruct-v0.2 · LongBench avg, B=1024L | average score: 41.13 -> 41.32 | Improved |
| use hit-rate eviction instead of cumulative recent-window attention (HitKV vs SnapKV) | SnapKV Llama3-8B-Instruct · LongBench avg, B=128L | average score: 35.69 -> 37.16 | Improved |
| use hit-rate eviction instead of cumulative recent-window attention (HitKV vs SnapKV) | SnapKV Llama3-8B-Instruct · LongBench avg, B=1024L | average score: 41.16 -> 41.3 | No clear effect |
| use hit-rate eviction instead of cumulative recent-window attention (HitKV vs SnapKV) | SnapKV Mistral-7B-Instruct-v0.2 · LongBench avg, B=128L | average score: 35.17 -> 36.29 | Improved |
| use hit-rate eviction instead of cumulative recent-window attention (HitKV vs SnapKV) | SnapKV Mistral-7B-Instruct-v0.2 · LongBench avg, B=1024L | average score: 41.54 -> 41.63 | No clear effect |
| use hit-rate eviction instead of SnapKV scoring | SnapKV Llama3-8B-Instruct · Needle-in-a-Haystack 8K, B=64L | accuracy: 69.3 -> 70.6 | Improved |
| use hit-rate eviction instead of SnapKV scoring | SnapKV Mistral-7B-Instruct-v0.2 · Needle-in-a-Haystack 32K, B=64L | accuracy: 59.5 -> 77.0 | Improved |
| compress KV cache to fixed budget 128L with HitKV vs full cache | Mistral-7B-Instruct-v0.2 · Decoding latency, 24k input | speedup: 4× inference acceleration at 24k input length | Improved |
| add hit-rate (activation-frequency) eviction in place of cumulative attention scores to P-HitKV for PG19 generation | P-HitKV · PG19 perplexity | perplexity | Hurt |
Closest papers by meaning
- SnapKV: LLM Knows What You are Looking for Before Generation (NeurIPS 2024 · similarity 0.90)
- CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration (ICML 2025 · similarity 0.89)
- CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences (ICLR 2025 · similarity 0.89)
- AttentionPredictor: Temporal Patterns Matter for KV Cache Compression (NeurIPS 2025 · similarity 0.88)
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.88)
- Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
