Research map / Large language models / Reasoning and chain of thought

AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoTool, a two-phase training framework (SFT+RL then KL-regularized Plackett-Luce ranking over embedding-anchored tool selection) plus a 200k tool-selection dataset, letting LLM agents select from evolving toolsets.

Problem

LLM agents trained with agentic RL assume a fixed, predefined tool inventory and fail to generalize to new or evolving toolsets with unseen tools at inference time.

Approach

Selects tools by sampling from a softmax over negative squared distances between a predicted anchor embedding and frozen tool embeddings from the agent's own embedding layer, and refines selection with a KL-regularized PL-ranking objective that reduces to a cross-entropy loss over rollouts.

Key results

• AutoTool (Qwen3-8B) reaches 66.7 on AIME24 and 45.1 on HotpotQA, matching or exceeding GRPO (56.7, 38.6) and ARPO (66.7, 43.6) baselines.
• AutoTool (Qwen2.5-VL-7B) reaches 49.3 on MMSearch and 52.5 on V-Code, above ARPO (48.2 and 50.9).
• Restricting inference to 886 unseen tools drops AIME24 and HotpotQA performance by less than 2% relative to the full toolset.

Surprising findings

• Analysis shows AutoTool relies largely on unseen tools in correct trajectories, especially for search and multimodal tasks.
• AutoTool's selection nearly matches an oracle that is given the ground-truth tool, with differences under 1 point on several benchmarks.

Limitations

• Training uses only 460 seen tools, so generalization claims rest on inference-time expansion to unseen tools.
• Training data rationales are generated by DeepSeek-R1 and filtered by LLM-as-a-judge, so quality depends on the teacher and judge.
• Some reported comparisons are against baselines from different model sizes and settings, and the Qwen2.5-VL-7B AutoTool scores are lower than baselines on V-Chart.

Experimental setup

Datasets: AIME24 AIME25 GPQA-Diamond HotpotQA 2WikiMultiHopQA Bamboogle MMSearch ChartGemma GSM8K MBPP HumanEval LiveCodeBench

Benchmarks: AIME24 AIME25 GPQA-Diamond HotpotQA 2WikiMultiHopQA Bamboogle MMSearch V-Chart V-Math V-Code LiveCodeBench-V2 MedQA

Metrics: accuracy tool-selection hit rate

Baselines: GPT-4o Qwen2.5-VL-72B-Instruct Qwen2.5-Math-72B-Instruct QwQ-32B DeepSeek-R1-Distill-Qwen-14B Toolformer ToolGen v-ToolRL Search-R1 ReTool SFT GRPO

Builds on: GRPO ARPO Search-R1 ReTool Plackett-Luce ranking

Closest papers by meaning

• AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.95)
• ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (ICLR 2026 · similarity 0.91)
• ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
• Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning (ICML 2026 · similarity 0.91)
• Towards Robust Multi-Modal Reasoning via Model Selection (ICLR 2024 · similarity 0.91)
• Tools as Continuous Flow for Evolving Agentic Reasoning (NeurIPS 2026 · similarity 0.90)