Research map / Multi-agent systems and games / LLM agents
KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KVComm, a training-free, online anchor-pool method that reuses and offset-corrects KV-caches of shared context across agents in LLM multi-agent systems, reporting up to 7.8x TTFT speedup.
Problem
Multi-agent LLM systems repeatedly prefill overlapping context across agents; standard KV-cache reuse fails when the same shared text sits behind different prefixes because of offset variance in the caches.
Approach
Treat cross-prefix KV reuse as approximate translation: store observed KV deviations for shared segments as anchors in an online-updated pool, match anchors by embedding similarity and length, and approximate new offsets by softmax-weighted interpolation, with RoPE de-rotation for keys.
Key results
- On GSM8K with 2-5 agents, KVComm keeps accuracy within about 2% of the original (79.6-81.7%) while achieving 67.6-87.6% reuse rates across benchmarks.
- KVComm reaches up to 7.82x TTFT speedup for the fifth agent (TTFT ~430ms to ~55ms per abstract), versus CacheBlend's accuracy collapse to 57.1% on GSM8K with five agents.
- On HumanEval, KVComm keeps Pass@1 at 81.4-83.2% while CacheBlend drops to 21-33%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KVComm instead of CacheBlend | MMLU, 4-agent multi-agent inference · MMLU | accuracy: 65.4 -> 68.0 | Helped |
| Use KVComm instead of no cache reuse | MMLU, 4-agent multi-agent inference · MMLU | accuracy: 68.0 -> 68.0 | No clear effect |
| Use KVComm instead of CacheBlend | GSM8K, 4-agent multi-agent inference · GSM8K | accuracy: 65.1 -> 80.6 | Helped |
| Use KVComm instead of no cache reuse | GSM8K, 4-agent multi-agent inference · GSM8K | accuracy: 82.1 -> 80.6 | Hurt |
| Use KVComm instead of CacheBlend | HumanEval, 4-agent multi-agent inference · HumanEval | Pass@1: 30.4 -> 83.2 | Helped |
| Use KVComm instead of no cache reuse | HumanEval, 4-agent multi-agent inference · HumanEval | Pass@1: 84.5 -> 83.2 | Hurt |
| Add positional alignment via key rotation (ablation) | KVComm on MMLU, 4 agents · MMLU | accuracy | Helped |
| Add placeholder KV-cache offset (ablation) | KVComm on MMLU, 4 agents · MMLU | accuracy | Helped |
| Add prefix segment KV-cache offset (ablation) | KVComm on MMLU, 4 agents · MMLU | accuracy | Helped |
| Combine key rotation and prefix offset without placeholder offset (ablation) | KVComm on MMLU, 4 agents · MMLU | accuracy: 62.1% | Hurt |
| Set entropy threshold to gamma=0.1 for conservative reuse (ablation) | KVComm on GSM8K, 4 agents · GSM8K | accuracy: 1% | Helped |
| Relax the reuse threshold (ablation) | KVComm on GSM8K, 4 agents · GSM8K | reuse rate: up to 98.2% | Helped |
Surprising findings
- CacheBlend's fixed 80% reuse policy causes large accuracy collapse on GSM8K (82.0% to 57.1%) and HumanEval (Pass@1 ~21-33%), showing fixed reuse policies break under offset variance.
Limitations
- Only evaluated on text inputs and homogeneous agents sharing the same model weights.
- Decoding latency is not accelerated by the method.
- Accuracy depends on request order and hyperparameters such as gamma and anchor pool size.
Experimental setup
Datasets: MMLU GSM8K HumanEval
Benchmarks: MMLU GSM8K HumanEval
Metrics: accuracy Pass@1 reuse rate TTFT speedup
Baselines: Original (no cache reuse) CacheBlend
Builds on: CacheBlend GPTSwarm AgentPrune DroidSpeak vLLM
Closest papers by meaning
- RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.92)
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.91)
- KVComm: Enabling Efficient LLM Communication through Selective KV Sharing (ICLR 2026 · similarity 0.91)
- LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents (ICML 2026 · similarity 0.91)
- AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems (NeurIPS 2026 · similarity 0.90)
- Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.