Research map / Architectures and efficiency / Attention and transformers

AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AgentKVShift, a training-free method that recomputes a small probe set of tokens per memory unit, estimates a shared layer-wise KV residual offset, and corrects all reused tokens, enabling near full-recompute quality for agentic memory at 10-30% refresh.

Problem

KV cache reuse for retrieved agentic memory units (LLM-curated notes, summaries, tags, graphs) degrades substantially under existing token-selection reuse methods designed for raw RAG passages, making prefill expensive.

Approach

Decomposes the per-memory KV reuse residual into a shared chunk-level offset plus small token-wise fluctuations; estimates the offset from a small probe set of recomputed tokens and applies a weighted mean-shift correction to all reused (non-probe) tokens, rather than only choosing which tokens to recompute.

Key results

• On LoCoMo with AMem and Qwen2.5-3B at r=0.1, AgentKVShift reaches 0.319 F1 vs 0.339 full recompute, while CacheBlend and ProphetKV drop 47.5% and 63.1%.
• On AMA-Bench-Recall with Qwen3-32B at r=0.3, AgentKVShift achieves 0.284 weighted F1 vs 0.296 full recompute and 0.279 accuracy vs 0.287.
• Under KIVI 2-bit at r=0.1 on LoCoMo, AgentKVShift retains 0.203 F1 while CacheBlend and ProphetKV fall below 0.08.

Surprising findings

• The probe-estimated offset absorbs average quantization error, so the method remains robust under 2-bit quantization where baselines collapse.
• Speedups at r=0.1 are similar across all three reuse methods, as the correction cost is negligible relative to recomputation.

Limitations

• Gap to full recompute is larger on cross-chunk reasoning capabilities (causal inference, state updating, state abstraction), per the authors.
• Theoretical guarantees rely on sub-Gaussian and mean-zero residual assumptions that are only empirically checked; the bound is for a single attention head/query setting.
• Evaluation limited to LoCoMo with 3B-7B models and AMA-Bench with one 32B model; judge-based accuracy depends on GPT-4o.

Experimental setup

Datasets: LoCoMo AMA-Bench

Benchmarks: LoCoMo AMA-Bench-Recall

Metrics: F1 ROUGE-1 BLEU-1 LLM-judge accuracy prefill speedup

Baselines: CacheBlend ProphetKV Full Recompute

Builds on: CacheBlend ProphetKV vLLM prefix caching KIVI

Closest papers by meaning

• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.90)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.90)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.90)
• AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.89)
• A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.89)
• MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging (NeurIPS 2026 · similarity 0.89)