Research map / Architectures and efficiency / Attention and transformers

Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

NeurIPS 2026 · Architectures and efficiency › Attention and transformers · approach: KV cache eviction and compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a closed-form, gradient-free per-head ridge mapper that transfers KV caches between same-shape models in a family, retaining 73–98% accuracy on four of six pairs at 2.7–25× lower latency than re-prefill.

Problem

Swapping between models in a family forces the receiver to re-prefill the context from scratch; reusing the source's KV cache requires mapping it into the target's KV format, which prior methods do with gradient-trained fusers or architectural constraints.

Approach

Cross-model KV across matched-KV pairs has substantial linear structure, so a per-(target layer, head) ridge regression over the top-k most predictive source layers, fit on 500 FineWeb-Edu sequences with RoPE stripped from keys, maps source KV into target KV without gradient training.

Key results

• On Qwen3 14B→32B, the ridge mapper retains 97.6% of HellaSwag accuracy and 96.3% floor-normalized average across five benchmarks.
• Four of six matched-KV pairs retain 73–98% average accuracy; Ministral 3B→14B and 8B→14B fall to 42–44%.
• An MLP replacement lifts HellaSwag retention by +24.3 to +36.8 pp on the two Ministral failure pairs; the mapper is 2.7–25× faster than re-prefill at 32K tokens on small-to-large transfer.

Surprising findings

• Calibration R^2 does not predict cross-pair retention: Llama 3.1 8B→70B and Ministral 3B→8B both fit at R^2_K=0.84 yet differ sharply in outcomes.
• Attention-output cosine correlates with HellaSwag retention (Pearson r=+0.57) while calibration R^2 shows r=-0.20.

Limitations

• Calibration uses only FineWeb-Edu; domain shift to other fields is not bounded.
• Per-pair k is selected on the same log-likelihood benchmarks that are reported, and only matched-KV pairs in three families are tested.
• Two of six pairs degrade sharply, and the paper does not establish a predictor for failure before fitting.

Experimental setup

Datasets: FineWeb-Edu WikiText-2 CoQA CodeAlpaca Wikipedia

Benchmarks: ARC-Challenge HellaSwag WinoGrande MMLU GSM8K WikiText-2 CoQA PIQA BoolQ ARC-Easy

Metrics: accuracy retention floor-normalized retention perplexity F1 R^2 attention-output cosine latency

Baselines: C2C LatentAlign IAM DroidSpeak re-prefill

Builds on: RoPE C2C DroidSpeak

Closest papers by meaning

• Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation (NeurIPS 2026 · similarity 0.86)
• MiniPIC: Flexible Position-Independent Caching in <100LOC (NeurIPS 2026 · similarity 0.86)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.85)
• KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache (ICML 2024 · similarity 0.85)
• KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference (ICML 2025 · similarity 0.85)
• SqueezeAttention: 2D Management of KV-Cache in LLM Inference via Layer-wise Optimal Budget (ICLR 2025 · similarity 0.85)