Research map / Architectures and efficiency / Attention and transformers
LookaheadKV: Fast and Accurate KV Cache Eviction by Glimpsing into the Future without Generation
Jinwoo Ahn, Ingyu Seong, Akhil Kedia, Junhan Kim, Hyemi Jang, Kangwook Lee, Yongkweon Jeon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add lookahead LoRA on all linear layers to embedding-only lookahead tokens (ablation) | LookaheadKV · LongBench budget 64, LLaMA3.2-1B, 32 lookahead | avg score: 26.40 -> 27.12 | Improved |
| restrict LoRA to Q and V instead of all linear layers (ablation) | LookaheadKV · LongBench budget 64, LLaMA3.2-1B | avg score: 27.12 -> 26.85 | Hurt |
| increase lookahead size beyond 32 (ablation) | LookaheadKV · LongBench budget 64, LLaMA3.2-1B | avg score: 27.12 -> 27.0 (128 tokens) | No clear effect |
| increase lookahead size from 4 to 32 (ablation) | LookaheadKV all · LongBench budget 64, LLaMA3.2-1B | avg score: 26.60 -> 27.12 | Improved |
| increase lookahead size beyond 32 (ablation) | LookaheadKV · inference overhead | TTFT overhead (%): 4.96 -> 11.0 | Hurt |
| add SnapKV-style suffix window to lookahead importance scores (ablation) | LookaheadKV · LongBench budget 64, LLaMA3.2-1B | avg score: 29.10 -> 28.52 | Hurt |
| replace draft generation with learned lookahead tokens | LAQ · TTFT overhead at 32K | eviction overhead: 14.5x lower | Improved |
| use learned lookahead instead of draft-based surrogate | LAQ · LongBench budget 128 greedy, LLaMA3.1-8B | avg score: 46.61 -> 47.72 | Improved |
| use learned lookahead instead of draft-based surrogate | LAQ · MT-Bench budget 256, LLaMA3.2-3B | judge score: 7.43 -> 7.30 | Hurt |
| evaluate at higher sampling temperature T=0.8 | LookaheadKV and baselines · LongBench budget 128, LLaMA3.1-8B | avg score: 47.72 -> 45.81 | Hurt |
| train with shorter context (2K, 4K, 8K) (ablation) | LookaheadKV · RULER, LLaMA-3B | avg score | Hurt |
Closest papers by meaning
- Taming the Fragility of KV Cache Eviction in LLM Inference (ICLR 2026 · similarity 0.92)
- ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution (ICML 2026 · similarity 0.92)
- Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference (NeurIPS 2025 · similarity 0.92)
- Learning to Evict from Key-Value Cache (ICML 2026 · similarity 0.91)
- CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences (ICLR 2025 · similarity 0.91)
- DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
