Research map / Large language models / Retrieval and knowledge
AvaTaR: Optimizing LLM Agents for Tool Usage via Contrastive Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AvaTaR, which optimizes an LLM agent's prompts and tool-use action sequences via a comparator LLM that performs batch-wise contrastive reasoning over positive and negative training queries, evaluated on STaRK, Flickr30K-Entities retrieval and three QA benchmarks.
Problem
LLM agents that use tools for multi-step retrieval and QA require hand-engineered prompts that are brittle and labor-intensive; existing LLM-as-optimizer methods fail to refine tool-usage strategies and generalize poorly.
Approach
A comparator LLM contrasts batches of positive (well-performing) and negative (poorly-performing) queries to produce holistic instructions (a robust "gradient") that update an actor LLM's prompts and action code, with a memory bank of top-5 past action sequences.
Key results
- On STaRK, AvaTaR achieves average ~14-15.6% relative Hit@1 improvement over the best baseline (e.g., Amazon Hit@1 49.87 vs 42.79 for Reflexion).
- On Flickr30K-Entities, AvaTaR boosts Hit@1 from 5.1% to 28.6% in 25 iterations.
- On QA tasks, AvaTaR reaches 53% EM on HotpotQA and 84% on ArxivQA, outperforming Reflexion and ExpeL; average ~13% relative gain on QA datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the comparator module to AvaTaR (ablation) | AvaTaR · STaRK Amazon | Hit@1: 40.92 -> 49.87 | Helped |
| Add the comparator module to AvaTaR (ablation) | AvaTaR · STaRK MAG | Hit@1: 33.25 -> 44.36 | Helped |
| Add the comparator module to AvaTaR (ablation) | AvaTaR · STaRK Prime | Hit@1: 8.82 -> 18.44 | Helped |
| Add the comparator module to AvaTaR (ablation) | AvaTaR · HotpotQA | exact match: 41.0% -> 53.0% | Helped |
| Add the comparator module to AvaTaR (ablation) | AvaTaR · ArxivQA | LLM judge score: 73.0% -> 84.0% | Helped |
| Add the comparator module to AvaTaR (ablation) | AvaTaR · ToolQA SciREX-hard | LLM judge score: 13.3% -> 23.3% | Helped |
| Optimize prompts through AvaTaR for 25 iterations | AvaTaR · Flickr30K-Entities | Hit@1: 5.1% to 28.6% | Helped |
| Optimize prompts through AvaTaR for 25 iterations | AvaTaR · STaRK Prime | Recall@20: 30.3% to 39.3% | Helped |
| Set the positive and negative query thresholds to ℓ=0.6 and h=0.5 instead of the default ℓ=h=0.5 (ablation) | AvaTaR · STaRK Amazon | Hit@1: 49.87 -> 50.45 | Helped |
Surprising findings
- Reflexion agents overfit to specific image data on Flickr30K-Entities, attempting infeasible actions such as extracting hat colors from images without hats.
- Removing the comparator (AvaTaR-C) drops performance substantially on some STaRK datasets, e.g., Prime Hit@1 from 18.44 to 8.82.
Limitations
- Scalability to hundreds of tools and long contexts raises latency and cost.
- Performance may degrade on queries requiring unfamiliar tool combinations.
- High computational and API cost; ExpeL comparison on retrieval limited to sampled test sets due to cost.
Experimental setup
Datasets: STaRK-Amazon STaRK-MAG STaRK-Prime Flickr30K-Entities HotpotQA ArxivQA ToolQA
Benchmarks: STaRK-Amazon STaRK-MAG STaRK-Prime Flickr30K-Entities HotpotQA ArxivQA ToolQA
Metrics: Hit@1 Hit@5 Recall@20 MRR exact match LLM judge score
Baselines: DPR QAGNN ada-002 multi-ada-002 ReAct Reflexion ExpeL Retroformer
Builds on: ReAct Reflexion ExpeL Retroformer
Closest papers by meaning
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.88)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.87)
- Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.86)
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.86)
- Fleet of Agents: Coordinated Problem Solving with Large Language Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.