Research map / Large language models / Agents, code and math
ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
Hongjin SU, Shizhe Diao, Ximing Lu, Mingjie Liu, Jiacheng Xu, Xin Dong, Yonggan Fu, Peter Belcak, Hanrong Ye, Hongxu Yin, Yi Dong, Evelina Bakhturina and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| give Claude Opus 4.1 basic tools | Claude Opus 4.1 · HLE | accuracy: 11.7 -> 19.8 | Improved |
| give Claude Opus 4.1 basic tools | Claude Opus 4.1 · FRAMES | accuracy: 58.2 -> 63.5 | Improved |
| add specialized and generalist LLMs as tools | Qwen3-235B-A22B · HLE | accuracy: 14.0 -> 32.8 | Improved |
| add specialized and generalist LLMs as tools | Qwen3-235B-A22B · FRAMES | accuracy: 39.5 -> 74.2 | Improved |
| add specialized and generalist LLMs as tools (prompted orchestrator) | GPT-5 · HLE | accuracy: 35.1 -> 21.2 | Hurt |
| add specialized and generalist LLMs as tools (prompted orchestrator) | GPT-5 · FRAMES | accuracy: 74.0 -> 57.5 | Hurt |
| add specialized and generalist LLMs as tools | Qwen3-8B · HLE | accuracy: 4.7 -> 30.6 | Improved |
| RL-train Qwen3-8B as orchestrator (ToolOrchestra) vs GPT-5 with basic tools | Orchestrator-8B · HLE | accuracy: 35.1 -> 37.1 | Improved |
| RL-train orchestrator vs GPT-5 with basic tools | Orchestrator-8B · tau2-Bench | accuracy: 77.7 -> 80.2 | Improved |
| RL-train orchestrator vs GPT-5 with basic tools (cost) | Orchestrator-8B · HLE+FRAMES avg | cost (US cents): 30.2 -> 9.2 | Improved |
| evaluate with tool set unseen in training | Orchestrator-8B · HLE | accuracy: 22.0 vs 17.8 (Claude Opus 4.1) | Improved |
| evaluate under unseen pricing configuration (deepinfra) | Orchestrator-8B · HLE | accuracy: 37.1 -> 36.9 | No clear effect |
Closest papers by meaning
- A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.91)
- AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning (ICML 2026 · similarity 0.91)
- ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
- Joint Optimization of Tool Creation and Use for Large Language Model Agents (NeurIPS 2026 · similarity 0.91)
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.89)
- Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
