Research map / Large language models / Agents, code and math
Learning Evolving Tools for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ToolEVO, an MCTS-based framework where LLMs explore a dynamic tool environment, self-reflect on errors and update tool descriptions, then fine-tune on successful trajectories; also introduces the ToolQA-D benchmark for tool variability.
Problem
LLM tool use degrades when APIs change in names, parameters, or response formats after training, since static fine-tuning and prompts with outdated tool manuals fail to match the deployed server APIs.
Approach
Use Monte Carlo Tree Search to let the LLM actively interact with a dynamic environment, reflecting on invocation and deprecation errors and writing updated tool usage into the prompt via an UpdateTool system tool; then SFT on successful trial-and-error trajectories so the model learns adaptation rather than memorizing static invocation patterns.
Key results
- On the OOD dynamic environment, ToolEVO (Qwen2-7B) averages 65.3 Easy / 27.7 Hard, versus 47.8 / 16.9 for Static-SFT (Qwen2-7B).
- On the in-domain dynamic environment, ToolEVO (Llama3-8B) reaches 66.2 Easy average versus 37.2 for Static-SFT (Llama3-8B), and exceeds Llama3-70B-Instruct (52.4).
- Ablations on the OOD setting show removing self-reflection drops the Easy average from 61.6 to 41.7 and removing tool update drops it to 55.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on trial-and-error experiences from dynamic environments rather than static-environment tool trajectories | Llama3-8B · ToolQA-D average, in-domain dynamic environment (Easy) | accuracy: 37.2 -> 66.2 | Helped |
| train on trial-and-error experiences from dynamic environments rather than static-environment tool trajectories | Qwen2-7B · ToolQA-D average, in-domain dynamic environment (Hard) | accuracy: 17.3 -> 32.1 | Helped |
| train on trial-and-error experiences from dynamic environments rather than static-environment tool trajectories | Llama3-8B · ToolQA-D average, OOD dynamic environment (Easy) | accuracy: 20.4 -> 61.6 | Helped |
| train on trial-and-error experiences from dynamic environments rather than static-environment tool trajectories | Qwen2-7B · ToolQA-D average, OOD dynamic environment (Hard) | accuracy: 16.9 -> 27.7 | Helped |
| train on trial-and-error experiences focused on tool variability instead of static tool usage | Llama3-8B · ToolQA-D average, static environment (Easy) | accuracy: 70.2 -> 66.7 | Mixed |
| add the tool-update module to summarize new tool usage and update the prompt (ablation) | ToolEVO · ToolQA-D average, OOD dynamic environment (Easy) | accuracy: 55.4 -> 61.6 | Helped |
| add the tool-update module to summarize new tool usage and update the prompt (ablation) | ToolEVO · ToolQA-D average, OOD dynamic environment (Hard) | accuracy: 21.9 -> 28.3 | Helped |
| add self-reflection for invocation errors (ablation) | ToolEVO · ToolQA-D average, OOD dynamic environment (Easy) | accuracy: 41.7 -> 61.6 | Helped |
| add self-reflection for invocation errors (ablation) | ToolEVO · ToolQA-D average, OOD dynamic environment (Hard) | accuracy: 15.5 -> 28.3 | Helped |
Surprising findings
- Static-SFT, despite strong static-environment performance, degrades sharply under tool changes, suggesting it develops over-trust in prompt-provided tool usage.
- Special-character insertion in API names hurts LLM tool use more than most other variations, while response-format changes have minimal impact.
Limitations
- Evaluated only on ToolQA-D with 12 tools across 7 datasets, a relatively small tool set.
- Uses only SFT on successful trajectories; failed trajectories and preference learning are not explored.
- API changes are synthetic, generated by GPT-4 rather than from real API version histories.
Experimental setup
Datasets: ToolQA
Benchmarks: ToolQA-D
Metrics: accuracy (success rate)
Baselines: ChatGPT GPT-4 GPT-4o GPT-4o-mini Claude-3.5-Sonnet Llama3-70B-Instruct Llama3-8B-Instruct Qwen2-72B-Instruct Qwen2-7B-Instruct Static-SFT
Builds on: ToolQA ReAct PUCT AlphaGo Zero
Closest papers by meaning
- From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions (ICLR 2025 · similarity 0.90)
- ToolACE: Winning the Points of LLM Function Calling (ICLR 2025 · similarity 0.89)
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (ICLR 2024 · similarity 0.89)
- ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- Joint Optimization of Tool Creation and Use for Large Language Model Agents (NeurIPS 2026 · similarity 0.89)
- ToolGen: Unified Tool Retrieval and Calling via Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.