Research map / Large language models / Agents, code and math
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DRAFT, a trial-and-error framework where an LLM explorer, analyzer, and rewriter iteratively refine human-written tool documentation using real tool-call feedback, with diversity-promoting exploration and tool-adaptive termination.
Problem
Human-centric tool documentation is incomplete, redundant, and inaccurate for LLMs, and manual revision does not scale or track evolving tools.
Approach
Automatically rewrite tool documentation through iterative exploration of real tool executions: an explorer generates diverse queries (embedding-similarity constraint with self-reflection), an analyzer proposes natural-language revision suggestions, and a rewriter updates the documentation and suggests next explorations, stopping per tool when BLEU and embedding similarity show convergence.
Key results
- On RestBench-TMDB with GPT-4o, DRAFT raises CP% from 71.00 (ReAct) to 88.00 and Win% to 71.00.
- On ToolBench with GPT-4o-mini, DRAFT documentation reaches 47.00 CP% versus 37.00 for DFSDT and 42.00 for EasyTool, and surpasses raw-doc GPT-4o (37.00 ReAct).
- Human evaluation on RestBench shows DRAFT documentation preferred for completeness 40% vs 16% raw, and accuracy 30% vs 0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o-mini · RestBench-TMDB | CP%: 56.00 -> 62.00 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o-mini · RestBench-Spotify | CP%: 35.08 -> 43.85 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o-mini · ToolBench | CP%: 42.00 -> 47.00 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | Llama-3-70B · RestBench-TMDB | CP%: 76.00 -> 86.00 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | Llama-3-70B · RestBench-Spotify | CP%: 63.15 -> 66.66 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | Llama-3-70B · ToolBench | CP%: 46.00 -> 53.00 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o · RestBench-TMDB | CP%: 79.00 -> 88.00 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o · RestBench-Spotify | CP%: 64.91 -> 70.17 | Helped |
| Refine tool documentation through iterative interaction feedback (DRAFT versus strongest baseline) | GPT-4o · ToolBench | CP%: 45.00 -> 51.00 | Helped |
| Add diversity-promoting exploration (ablation) | DRAFT with GPT-4o · RestBench-TMDB | CP%: 84.00 -> 88.00 | Helped |
| Add diversity-promoting exploration (ablation) | DRAFT with GPT-4o · RestBench-TMDB | Win%: 69.00 -> 71.00 | Helped |
| Add tool-adaptive termination (ablation) | DRAFT with GPT-4o · RestBench-TMDB | CP%: 80.00 -> 88.00 | Helped |
Surprising findings
- Performance declines after a certain number of iterations, attributed to redundant documentation growth and overfitting, motivating the adaptive termination.
- Documentation refined using GPT-4o feedback also improves Llama-3-70B and GPT-4o-mini, indicating cross-model generalization.
Limitations
- Evaluation limited to a subset of ToolBench (I3-Instruction) due to budget constraints.
- Win% judged by ChatGPT-based evaluator rather than humans for main results.
- Performance declines with too many iterations, so termination threshold tuning is needed.
Experimental setup
Datasets: ToolBench (I3-Instruction subset) RestBench (TMDB, Spotify)
Benchmarks: RestBench-TMDB RestBench-Spotify ToolBench I3-Instruction
Metrics: CP% Win% NDCG@1 NDCG@10 human pairwise preference (completeness, conciseness, accura
Baselines: ReAct DFSDT EasyTool
Builds on: ReAct DFSDT EasyTool
Closest papers by meaning
- Learning Evolving Tools for Large Language Models (ICLR 2025 · similarity 0.90)
- MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use (ICLR 2024 · similarity 0.88)
- Automated Creation of Reusable and Diverse Toolsets for Enhancing LLM Reasoning (AAAI 2025 · similarity 0.88)
- ToolGen: Unified Tool Retrieval and Calling via Generation (ICLR 2025 · similarity 0.88)
- Large Language Models as Tool Makers (ICLR 2024 · similarity 0.88)
- MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.