Research map / Large language models / Reasoning and chain of thought

Joint Optimization of Tool Creation and Use for Large Language Model Agents

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SMITH, a DAPO-based RL framework that jointly trains one LLM to write reusable Python tools with JSON schemas (build task) and to invoke them (use task), using execution, judge, and format rewards on Reasoning-Gym.

Problem

Inference-time tool-creation methods leave the tool writer decoupled from the tool user and do not explicitly optimize tool quality or reusability; training a single model to both create and use tools raises reward decomposition and circular-evaluation challenges.

Approach

A single policy is trained with DAPO on two balanced task streams: building a Python function plus JSON schema evaluated on held-out harder questions by a periodically synced evaluator copy, and using only the schema to answer queries. Format, execution accuracy, and an LLM judge score are kept as separate reward axes.

Key results

• SMITH (Qwen3-4B) reaches 79.9 macro accuracy on held-out Reasoning-Gym tasks, above CRAFT (76.5) and ReTool distilled (63.2).
• On TabMWP-Hard SMITH scores 40.4 versus TroVE's 36.4; on GQA it reaches 42.6, second to LATM distilled from GPT-4.1 (56.0).
• BFCL v4 accuracy rises from 45.1 to 48.6 on Qwen3-4B and from 43.3 to 55.8 on Qwen3-8B.

Surprising findings

• Self-judging with the policy's own model improves held-out RG (81.7 to 85.9) but degrades GQA (28.7 to 16.3).
• Tools written by the 4B policy let a frozen 350M model match a 30B tool writer on held-out RG (42.9 vs 41.5).

Limitations

• Policies limited to at most 8B parameters and the judge is a 30B-A3B model, so scaling behavior beyond this is untested.
• No formal safety certification for sandboxed execution of generated Python, and no human evaluation of schema readability.
• Self-judge variant trades off OOD GQA performance, indicating judge dependence.

Experimental setup

Datasets: Reasoning-Gym TabMWP GQA BFCL v4

Benchmarks: Reasoning-Gym (Seen) Reasoning-Gym (Unseen) TabMWP-Hard GQA BFCL v4 (no-web)

Metrics: macro-average accuracy output tokens

Baselines: Standard CoT LATM CRAFT TroVE KTCE ReTool LATM distill GPT-4.1 Qwen3-30B-A3B LATM

Builds on: DAPO GRPO LATM TroVE

Closest papers by meaning

• ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.92)
• ToolRL: Reward is All Tool Learning Needs (NeurIPS 2025 · similarity 0.90)
• Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoning (ICLR 2026 · similarity 0.90)
• Are Tools Always Beneficial? Learning to Invoke Tools Adaptively for Dual-Mode Multimodal LLM Reasoning (ICML 2026 · similarity 0.90)
• AutoTool: Efficient Tool Selection for Large Language Model Agents (AAAI 2026 · similarity 0.90)
• AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning (ICML 2026 · similarity 0.90)