Research map / Multi-agent systems and games / LLM agents

LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LRAgent, a KV cache sharing framework for multi-LoRA LLM agents that shares the pretrained base value cache and stores adapter contributions in low-rank form, with a Flash-LoRA-Attention kernel for efficient reconstruction.

Problem

Multi-LoRA LLM agents sharing a backbone still build and store separate KV caches for the same long tool-augmented trajectories, causing redundant memory and prefill computation; existing KV sharing methods ignore the multi-LoRA structure.

Approach

Decompose the value cache into a shared base component (X W0) and an adapter component stored as a low-rank LR cache (X A); share the base cache across agents, and with a shared-A multi-LoRA architecture also share the LR cache, expanding it via B only when needed using a reordered FlashAttention-style kernel.

Key results

• BaseLRShared stays within 0.97 points (LLaMA-3.1-8B) and 1.43 points (Ministral-8B) of Non-Shared average accuracy, while FullShared drops up to 5.28 points.
• With Flash-LoRA-Attention, BaseLRShared achieves up to 2.46x throughput gain and up to 4.44x TTFT reduction, close to FullShared.
• BaseShared and BaseLRShared reduce KV cache memory by nearly 1/3 relative to Non-Shared on a 66.4k-token trace on Ministral-8B.

Surprising findings

• Adapter outputs are small in magnitude (27.3x and 14.77x smaller than base cache) and nearly orthogonal across agents, yet sharing the full cache causes accuracy drops.
• Sharing the LR cache without shared-A degrades accuracy sharply (e.g., BaseLRShared 36.25 vs 42.05 Non-Shared on LLaMA HotpotQA easy).

Limitations

• BaseLRShared requires shared-A multi-LoRA training and cannot be applied directly to already-deployed non-shared-A adapters.
• Evaluation limited to AutoAct-based agent setups on HotpotQA and ScienceQA with two 8B models.
• BaseShared still requires LR prefill and hidden-state computation over context processed by other agents, so its computation remains similar to non-shared.

Experimental setup

Datasets: HotpotQA ScienceQA AutoAct agent trajectories MATH

Benchmarks: HotpotQA ScienceQA

Metrics: accuracy system throughput (tokens/s) time-to-first-token (s) KV cache memory (GB)

Baselines: Non-Shared FullShared DroidSpeak ICaRus CacheBlend KVComm

Builds on: LoRA FlashAttention AutoAct DroidSpeak shared-A multi-LoRA

Closest papers by meaning

• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.91)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.90)
• KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.90)
• Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents (NeurIPS 2025 · similarity 0.90)
• AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems (NeurIPS 2026 · similarity 0.89)
• CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control (ICML 2026 · similarity 0.87)