Research map / Multi-agent systems and games / LLM agents

Tvcache: A Tool-Value Cache for Post-Training LLM Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TVCache, a stateful tool-value cache organized as a tool call graph with longest-prefix matching and selective sandbox snapshotting, to speed up tool execution in RL post-training of LLM agents.

Problem

Tool calls in RL post-training of LLM agents take seconds to minutes while GPUs sit idle; naive output caching is incorrect because tool outputs depend on stateful sandbox state built by prior calls.

Approach

Maintain a per-task tool call graph of observed tool-call sequences, serve a cache hit only on exact longest-prefix match of the full tool history (guaranteeing identical sandbox state), fork cached sandboxes on partial matches, and snapshot selectively only for expensive tool calls.

Key results

• Cache hit rates reach up to 70%; average hit rate 64.3% on EgoSchema, 33.11% on SkyRL-SQL, and 14-25% on terminal-bench.
• Median per-tool-call execution time reduced by up to 6.9x on terminal-bench (Qwen3-4B, medium tasks).
• Reward curves with and without caching closely match across all workloads, with no degradation in post-training reward.

Surprising findings

• Larger models (Qwen3-14B) achieve higher hit rates on terminal-bench because they tend to repeat tool calls.
• Conventional signature-keyed caches for OpenAI API tool calls would return incorrect outputs across videos, so state tagging is required.

Limitations

• Hit rates on terminal-bench are modest (14-25% average), limiting end-to-end gains.
• Default conservative assumption that all tools mutate state reduces reuse unless developers annotate stateless tools.
• Exact-match keys miss near-duplicate tool arguments (e.g., minor text differences), so fuzzy matching is left to future work.

Experimental setup

Datasets: terminal-bench SkyRL-SQL EgoSchema

Benchmarks: terminal-bench (easy) terminal-bench (medium) SkyRL-SQL EgoSchema

Metrics: cache hit rate median per-tool-call execution time speedup reward curves P95 cache get latency memory overhead

Baselines: no caching (cacheless post-training) stateless tool-call cache keyed on signature

Builds on: GRPO veRL terminal-bench VideoAgent SkyRL-SQL

Closest papers by meaning

• KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.88)
• Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents (NeurIPS 2025 · similarity 0.87)
• CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control (ICML 2026 · similarity 0.87)
• RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse (ICML 2026 · similarity 0.87)
• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.86)
• Learning to Evict from Key-Value Cache (ICML 2026 · similarity 0.86)