Research map / Architectures and efficiency / Attention and transformers

PatchKV: Weight Space Compensation of KV Cache

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PatchKV, a training-free method that compensates compressed KV caches by computing a context-specific closed-form ridge-regression patch to MLP down-projections, merged into weights at context-loading time.

Problem

KV cache compression methods degrade sharply at aggressive budgets in the single-context, multi-query long-context setting, because all context information must be retained in the compressed cache.

Approach

Pair an off-the-shelf compressed KV cache with a context-specific weight patch, derived per block by ridge regression that aligns compressed-cache student MLP/attention activations to full-cache teacher activations on context-derived reference queries (repeat-context, self-study). The patch is merged into MLP down-projections so inference cost per query is unchanged.

Key results

• On SQuAD with KVzip, PatchKV recovers up to 73% of full-KV accuracy with no cache remaining, where the cache-only baseline collapses to near zero.
• Patches built from disjoint cache subsets improve over cache-only baseline (e.g., 54.95 to 85.39 on C_A with its own patch) in the cross-evaluation study.
• Low-rank (r=32) patches match full-rank performance within a small margin while reducing per-context patch storage by 99%.

Surprising findings

• On SCBench ICL.ManyShot, the baseline achieves its highest accuracy with no context KV retained, even exceeding full-KV.
• Compression itself improves Retrieve.MultiHop performance by removing distractors.

Limitations

• Relies on context-derived reference queries as proxies for unknown downstream queries; mismatch limits recovery under distribution shift.
• Requires one-time preprocessing and access to the full KV cache as teacher, so it pays off mainly when amortized over many queries or long decoding.
• Serving many context-specific patches concurrently needs additional mechanisms.

Experimental setup

Datasets: SCBench SQuAD NIAH GSM8K HotpotQA AIME 2025

Benchmarks: SCBench SQuAD NIAH GSM8K HotpotQA AIME 2025

Metrics: exact match accuracy Pass@1 ROUGE perplexity relative accuracy

Baselines: KVzip Attention Matching Doc-to-LoRA

Builds on: KVzip Attention Matching context-to-weight conversion (low-rank MLP update) Doc-to-LoRA

Closest papers by meaning

• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference (NeurIPS 2025 · similarity 0.92)
• KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction (NeurIPS 2025 · similarity 0.92)
• SCBench: A KV Cache-Centric Analysis of Long-Context Methods (ICLR 2025 · similarity 0.92)
• KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse (NeurIPS 2025 · similarity 0.91)
• RazorAttention: Efficient KV Cache Compression Through Retrieval Heads (ICLR 2025 · similarity 0.91)
• RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression (ICML 2025 · similarity 0.91)