Research map / Multi-agent systems and games / LLM agents

KVComm: Enabling Efficient LLM Communication through Selective KV Sharing

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KVComm, which lets an LLM sender share selected layers' KV pairs with a receiver LLM (selected via attention importance plus a Gaussian layer prior) instead of natural language or hidden states.

Problem

Inter-LLM communication in multi-agent systems is costly with natural language and lossy with hidden-state protocols (last-token bias, dilemma with all-token hidden states).

Approach

Share the KV pairs of a selected, possibly non-contiguous subset of sender layers, chosen by a score combining normalized average attention to context tokens with a Gaussian prior over layer depth; the receiver concatenates these KV pairs into its attention.

Key results

• On the Llama-3.2-3B pair, KVComm (0.7) matches or nearly matches Skyline on Tipsheets (0.96 vs 0.91) and outperforms it on HotpotQA (0.65 vs 0.73 is lower; sender-receiver mixed results noted) and TMATH (0.36 vs 0.36).
• KVComm (0.3) on the Llama-3.2-3B pair reaches 0.46 on HotpotQA vs 0.43 for NLD and 0.25 for AC (mean), while transmitting 30% of layers.
• Paper reports 2.5x to 6x computation reduction relative to Skyline and up to 3x communication reduction versus transmitting all KV pairs.

Surprising findings

• KVComm sometimes outperforms the Skyline upper bound on some datasets, which the authors attribute to complementary sender capabilities and regularization from selective sharing.
• Selecting a single calibration sample is reported as sufficient for robust layer selection.

Limitations

• Requires sender and receiver to share the same base architecture, since KV structures differ across families.
• Layer selection is fixed after calibration rather than context-adaptive.
• Gaussian prior is a simple heuristic; more data-driven priors are left to future work.

Experimental setup

Datasets: Countries Tipsheets HotpotQA QASPER MuSiQuest MultiFieldQA-en 2WikiMQA TMATH MMLU

Benchmarks: Countries Tipsheets HotpotQA QASPER MuSiQuest MultiFieldQA-en 2WikiMQA TMATH HotpotQA-E QASPER-E MuSiQuest-E SAMSum

Metrics: F1 ROUGE-L recall FLOPs memory

Baselines: Baseline Skyline NLD CIPHER AC (mean) AC (replace) AC (sum) random layer selection

Builds on: Natural Language Debate CIPHER Activation Communication DroidSpeak

Closest papers by meaning

• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.91)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.89)
• MOC: Multi-Order Communication in LLM-based Multi-Agent Systems (ICML 2026 · similarity 0.89)
• Communicating Activations Between Language Model Agents (ICML 2025 · similarity 0.88)
• A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability (NeurIPS 2026 · similarity 0.88)
• Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration (ICLR 2026 · similarity 0.88)