Research map / Multi-agent systems and games / LLM agents

AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AOrchestra, an orchestrator that dynamically creates tailored sub-agents via a four-tuple (Instruction, Context, Tools, Model), evaluated training-free on GAIA, Terminal-Bench 2.0, and SWE-Bench-Verified, plus SFT of Qwen3-8B and in-context cost-aware routing.

Problem

Existing sub-agent-as-tools designs either isolate context without specialization or use static, hand-engineered sub-agent roles, which hurts adaptability on long-horizon agentic tasks.

Approach

Model any agent as a four-tuple (Instruction, Context, Tools, Model) and have a pure orchestrator concretize this tuple at each step to spawn on-demand executors, with curated context and per-subtask model/tool selection; the orchestrator is framework-agnostic and learnable via SFT or prompt-level optimization for cost-aware routing.

Key results

• With Gemini-3-Flash, AOrchestra reaches 80.00 pass@1 on GAIA versus 66.06 for the strongest baseline (OpenHands), and 52.86 vs 34.29 on Terminal-Bench 2.0.
• SFT of Qwen3-8B as orchestrator raises GAIA accuracy from 56.97% to 68.48% with Gemini-3-Flash as sub-agent.
• In-context learning for cost-aware mixed-model routing improves accuracy from 72.12% to 75.15% while lowering average cost from $0.70 to $0.57.

Surprising findings

• Full-context inheritance for sub-agents performed worse than no context on GAIA in the context-control ablation (84.00 vs 86.00 average).
• Claude Code was excluded from GAIA because it is not suited to open-world QA, and DeepSeek-V3.2 had poor native compatibility with Claude Code.

Limitations

• Benchmark evaluations use subsets (70 of 89 Terminal-Bench tasks, 100 of 500 SWE-Bench-Verified tasks) due to cost.
• SFT uses only 2K trajectories with a simple approach; the fine-tuned Qwen3-8B still trails the Gemini-3-Flash orchestrator and raises per-task cost.
• Context ablation on GAIA uses only 50 samples.

Experimental setup

Datasets: GAIA validation TaskCraft Terminal-Bench 2.0 SWE-Bench-Verified

Benchmarks: GAIA Terminal-Bench 2.0 SWE-Bench-Verified

Metrics: pass@1 pass@3 average cost

Baselines: ReAct OpenHands Mini-SWE-agent Claude Code

Builds on: ReAct Claude Code sub-agents TaskCraft LLaMA-Factory

Closest papers by meaning

• MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts (NeurIPS 2026 · similarity 0.90)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.90)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs (NeurIPS 2026 · similarity 0.90)
• AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.89)
• AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.89)
• UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.89)