Research map / Large language models / Agents, code and math

ToolGen: Unified Tool Retrieval and Calling via Generation

ICLR 2025 · Large language models › Agents, code and math · approach: Long-horizon LLM agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ToolGen, which represents each of ~47k tools as a unique virtual token in an LLM's vocabulary so the model retrieves and calls tools via next-token generation, trained in three stages (memorization, retrieval, end-to-end agent tuning).

Problem

Tool learning with tens of thousands of tools cannot put all tools in context, and separate retrieval pipelines are inefficient and misaligned with execution; LLMs also lack intrinsic knowledge of tools.

Approach

Atomic indexing maps each tool/API to one new vocabulary token initialized from its name embedding; the LLM is fully fine-tuned to memorize tool docs, generate tool tokens from queries, and run multi-turn agent trajectories without an external retriever, using constrained beam search to restrict outputs to valid tools.

Key results

• In the multi-domain retrieval setting, ToolGen reaches NDCG@1 of 87.67 on I1 versus 72.31 for ToolRetriever.
• End-to-end with retriever, ToolGen achieves average SoPR 53.28 and SoWR 51.51, versus 51.55 and 49.70 for ToolLlama.
• With constrained beam search, ToolGen generates no nonexistent tools, while ToolLlama and GPT-3.5 still hallucinate tool names.

Measured effects reported in the paper

Surprising findings

• Without general instruction data, ToolGen's general NLP average drops to 26.94 from Llama-3's 63.76, near random on GSM8K.
• Removing retrieval training barely hurts end-to-end SoPR but collapses retrieval NDCG (e.g., I1 NDCG@1 falls to 10.17), and generalization to unseen tools is weaker than ToolLlama.

Limitations

• Cannot readily add new tools or handle major usage changes; requires retraining to incorporate new tools.
• General instruction-following ability is severely degraded without mixed instruction data.
• Generalization to unseen tools is weaker than ToolLlama in end-to-end evaluation.

Experimental setup

Datasets: ToolBench OpenHermes-2.5

Benchmarks: ToolBench StableToolBench ARC-C ARC-E CSQA Hellaswag Winogrande GSM8K

Metrics: NDCG@1 NDCG@3 NDCG@5 SoPR SoWR hallucination rate 3-shot accuracy

Baselines: BM25 EmbSim Re-Invoke IterFeedback ToolRetriever GPT-3.5 ToolLlama ToolLlama-2

Builds on: ToolBench Toolformer ToolkenGPT Agent-Flan StableToolBench

Closest papers by meaning

• ToolTree: Efficient LLM Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruning (ICLR 2026 · similarity 0.89)
• Joint Optimization of Tool Creation and Use for Large Language Model Agents (NeurIPS 2026 · similarity 0.89)
• ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
• Learning Evolving Tools for Large Language Models (ICLR 2025 · similarity 0.88)
• AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.88)
• From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions (ICLR 2025 · similarity 0.88)