Research map / Architectures and efficiency / Attention and transformers
EntroKV: Entropy-Guided Dynamic Budget Allocation for KV-Cache Compression
Wenhao Gao, Haoran Cao, Yueyan Li, YongGao Xiao, Caixia Yuan, Xiaojie Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add entropy-guided head-wise budget allocation to SnapKV | SnapKV · RULER-4K | score: 86.91 -> 89.33 | Improved |
| Add entropy-guided budget allocation to CaM | CaM · RULER-16K | score: 67.18 -> 91.57 | Improved |
| Add entropy-guided budget allocation to D2O | D2O · LongBench | score: 41.23 -> 45.89 | Improved |
| Add entropy-guided allocation to SnapKV on Qwen3-8B | SnapKV · LongBench | score: 48.25 -> 49.05 | Improved |
| Use entropy head-level allocation instead of PyramidKV or AdaKV layer/head allocation | SnapKV · LongBench (Llama-3.1-8B) | score: 47.79 -> 48.15 | Improved |
| Use LAVa or Task-KV allocation instead of uniform SnapKV | SnapKV · RULER-16K (Llama-3.1-8B) | score: 90.88 -> 87.12 | Hurt |
| Vary adaptivity alpha from 0 to 1 (mix of global baseline and instance entropy) (ablation) | EntroKV · LongBench (Llama-3.1-8B, Qwen2.5-7B) | accuracy | Mixed |
| Increase total budget ratio beta (ablation) | EntroKV · LongBench | accuracy | Improved |
| Use Entro-SnapKV at long context (128K) instead of SnapKV | SnapKV · RULER-128K (Llama-3.1-8B) | score: 72.68 -> 74.00 | Improved |
| Add entropy-guided allocation under few-shot MATH-500 and SQuAD | SnapKV, CaM, D2O · MATH-500, SQuAD (0-5 shot) | accuracy | Improved |
| Use entropy allocation instead of full cache at budget 0.4-0.5 | EntroKV on Llama · LongBench | accuracy | Improved |
Closest papers by meaning
- KV-COBRA: KV Cache Compression via Co-Optimized Bit-Rank Allocation (NeurIPS 2026 · similarity 0.91)
- CompilerKV: Risk-Adaptive KV Cache Compression via Offline Experience Compilation (NeurIPS 2026 · similarity 0.89)
- SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference (NeurIPS 2025 · similarity 0.89)
- SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.89)
- EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction (NeurIPS 2026 · similarity 0.88)
- AttentionPredictor: Temporal Patterns Matter for KV Cache Compression (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
