Research map / Large language models / Reasoning and chain of thought

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cache-to-Cache (C2C), which projects and fuses one LLM's KV-cache into another's via a learned gated fuser, enabling direct semantic communication without intermediate text.

Problem

Multi-LLM systems communicate via text, which loses semantic information in a low-bandwidth bottleneck and incurs slow token-by-token generation latency.

Approach

Treat the KV-cache as the communication medium: a neural cache fuser projects the Sharer's KV-cache into the Receiver's space, merges it residually with the Receiver's cache via per-layer learnable Gumbel-sigmoid gates, while both LLMs stay frozen.

Key results

• C2C averages 6.4-14.2% higher accuracy than individual models across Sharer settings.
• C2C outperforms text-to-text communication by about 3.1-5.4% average accuracy, e.g., 42.92 vs 41.03 on MMLU-Redux with Qwen2.5-0.5B Sharer.
• C2C gives an average 2.5x latency speedup over T2T; 445 ms vs 1596 ms total on MMLU-Redux (Qwen2.5-0.5B to Qwen3-0.6B).

Surprising findings

• Enriching a single layer's KV-cache with few-shot exemplars (Oracle) at fixed cache length improves accuracy over direct prompting (62.34 vs 58.42 on the oracle table), suggesting semantic enrichment rather than extra tokens drives gains.
• Qwen3-4B Base as Sharer often ignores instructions and yields 1.03% standalone MMLU-Redux accuracy, yet C2C still improves the Receiver.

Limitations

• Performance degrades when a much weaker Sharer provides noisy information to a stronger Receiver.
• Pairwise fusers; scaling to N models with O(N) training cost remains open.
• Replacing portions of cache at inference shows an initial accuracy drop due to train-test mismatch.

Experimental setup

Datasets: OpenHermes-2.5 MMLU auxiliary_train LongBench-E MMLU-Redux

Benchmarks: OpenBookQA MMLU-Redux ARC-Challenge C-Eval LongBench-E

Metrics: accuracy inference time effective rank PGR

Baselines: Text-to-Text (T2T) Query-level routing Receiver-only Sharer-only Single fine-tuning Identical-model C2C

Builds on: Text-to-text multi-agent communication query-level routing (matrix factorization) effective rank

Closest papers by meaning

• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference (NeurIPS 2025 · similarity 0.90)
• SCBench: A KV Cache-Centric Analysis of Long-Context Methods (ICLR 2025 · similarity 0.90)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.90)
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models (NeurIPS 2024 · similarity 0.90)
• LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models (ICML 2025 · similarity 0.89)
• Deliberation in Latent Space via Differentiable Cache Augmentation (ICML 2025 · similarity 0.89)