Research map / Multi-agent systems and games / LLM agents
Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Agentic Plan Caching, a test-time memory that extracts plan templates from completed agent runs, matches new tasks by extracted keyword, and adapts cached plans with a small LM to cut LLM agent cost and latency.
Problem
LLM agents that follow a Plan-Act loop incur high API cost and latency from repeated planning by expensive models, and existing context and semantic caching fail for data-dependent agent workloads.
Approach
Cache structured plan templates (not query-response pairs) keyed by an LLM-extracted task keyword, generalize them by removing context-specific details with a lightweight LLM filter, and adapt them with a small planner LM on cache hits, so the large planner is used only on misses.
Key results
- APC reduces average serving cost by 50.31% and latency by 27.28% while retaining 96.61% of accuracy-optimal performance across workloads.
- On GAIA with Open Deep Research, APC cuts cost from $69.02 to $16.27 (76.42% reduction) with accuracy 36.97% vs 37.58%.
- On FinanceBench, APC reaches 85.50% accuracy versus 72.00% for full-history caching at lower cost ($1.86 vs $1.99).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reuse and adapt cached plan templates with APC instead of no caching | Minion agent · QASPER | cost: $2.14 -> $0.78 | Helped |
| reuse and adapt cached plan templates with APC instead of no caching | Minion agent · QASPER | accuracy: 58.00% -> 57.00% | No clear effect |
| reuse and adapt cached plan templates with APC instead of no caching | Minion agent · AIME 2024 | cost: $1.14 -> $0.85 | Helped |
| reuse and adapt cached plan templates with APC instead of no caching | Minion agent · AIME 2025 | cost: $1.34 -> $0.81 | Helped |
| reuse and adapt cached plan templates with APC instead of no caching | Open Deep Research agent · GAIA | cost: $69.02 -> $16.27 | Helped |
| reuse and adapt cached plan templates with APC instead of no caching | Open Deep Research agent · GAIA | accuracy: 37.58% -> 36.97% | No clear effect |
| replace concise plan templates with full execution histories as cache examples (ablation) | agent plan caching · FinanceBench | accuracy: 85.50% vs. 72.00% | Hurt |
| increase cache size from 1 to 100 (ablation) | APC · FinanceBench | cost: $3.97 -> $1.86 | Helped |
| increase cache size from 1 to 100 (ablation) | APC · FinanceBench | accuracy: 92.00% -> 85.50% | Hurt |
| use fuzzy keyword matching with threshold >80% instead of exact matching (ablation) | APC · FinanceBench | cost: $1.86 -> $1.15 | Helped |
| use fuzzy keyword matching with threshold >80% instead of exact matching (ablation) | APC · FinanceBench | accuracy: 85.50% -> 83.00% | Hurt |
| lower fuzzy keyword threshold from >80% to >60% (ablation) | APC · FinanceBench | accuracy: 83.00% -> 77.00% | Hurt |
Surprising findings
- Query-based semantic caching at lower similarity thresholds degrades accuracy substantially due to false-positive hits.
- Smaller actor models such as Llama-3.2-3B often increase cost and reduce accuracy versus Llama-3.1-8B because they trigger more plan-act iterations.
Limitations
- Evaluated only on two-stage Plan-Act architectures; multi-agent cache consistency is unaddressed.
- Benefits diminish for highly dynamic workloads with frequent task variation; cold start adds overhead.
- Primarily evaluates cost; latency and throughput are reported only on a small FinanceBench microbenchmark.
Experimental setup
Datasets: FinanceBench QASPER TabMWP AIME 2024 AIME 2025 GAIA
Benchmarks: FinanceBench TabMWP QASPER AIME 2024 AIME 2025 GAIA
Metrics: cost accuracy latency hit rate
Baselines: Accuracy-Optimal Cost-Optimal Semantic Caching (GPTCache-style) Full-History Caching
Builds on: Minions ReAct GPTCache Case-Based Planning
Closest papers by meaning
- EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.90)
- LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents (ICML 2026 · similarity 0.90)
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.89)
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.89)
- KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.88)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.