Research map / Multi-agent systems and games / LLM agents

SkillOrchestra: Learning to Route Agents via Skill Transfer

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SkillOrchestra, which learns a reusable Skill Handbook of skills, agent profiles, and mode insights from execution traces, and uses it to route agents/models instead of RL-training the orchestrator.

Problem

Multi-turn routing and agent orchestration in compound AI systems: input-level routers ignore evolving task requirements, and RL-trained orchestrators are expensive to train, poorly adaptable, and prone to routing collapse onto one costly model.

Approach

Replace end-to-end RL routing policy learning with a learned Skill Handbook: fine-grained skills discovered from successful/failed trajectory contrasts, Beta-posterior agent competence per skill, and cost estimates; at inference, the orchestrator selects a mode and then the agent maximizing skill-weighted competence minus cost, with orchestrator-specific handbook granularity chosen by Pareto valid

Key results

• On QA benchmarks, SkillOrchestra reaches 47.4 average EM vs. Router-R1's 41.6 (+5.8), and SkillOrchestra+ reaches 51.6 (+10.0), with up to 22.5-point gains on math over Router-R1 at about half the cost.
• On FRAMES agent orchestration, SkillOrchestra achieves 84.3% accuracy at $72.7 vs. ToolOrchestra's 76.3% at $92.7.
• Router-R1 routes 98.02% of calls to LLaMA-3.1-70B (routing collapse), whereas SkillOrchestra spreads calls across models (e.g., Mixtral-8x22B 44.53%).

Surprising findings

• A handbook learned with Qwen2.5-3B transfers to other orchestrator backbones without retraining, improving every tested model, including Mixtral-8x22B (+14.8 points).
• Stronger proprietary orchestrators such as GPT-5 and Gemini 3 Pro are less cost-effective than SkillOrchestra despite comparable or higher per-token capability.

Limitations

• Relies on LLM-based discoverer and reflector (GPT-5) to create and refine skills, adding dependence on a strong proprietary model.
• Gains on agent orchestration are evaluated mainly on FRAMES with a 100-task ablation subset, limiting statistical strength of ablations.

Experimental setup

Datasets: Natural Questions TriviaQA PopQA HotpotQA 2WikiMultiHopQA Musique Bamboogle MATH AMC23 FRAMES

Benchmarks: Natural Questions TriviaQA PopQA HotpotQA 2WikiMultiHopQA Musique Bamboogle MATH AMC23 FRAMES

Metrics: exact match accuracy total cost (USD / cents)

Baselines: Router-R1 ToolOrchestra KNN Router MLP Router BERT Router RouterDC GraphRouter FrugalGPT Search-R1 RAG GPT-5 Claude Opus 4.5

Builds on: Router-R1 ToolOrchestra Search-R1 GRPO PPO

Closest papers by meaning

• SkillOS: Learning Skill Curation for Self-Evolving Agents (NeurIPS 2026 · similarity 0.88)
• Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Learning Latency-Aware Orchestration for Multi-Agent Systems (NeurIPS 2026 · similarity 0.87)
• Reward Modeling for Multi-Agent Orchestration (NeurIPS 2026 · similarity 0.87)
• MonoScale: Scaling Multi-Agent System with Monotonic Improvement (ICML 2026 · similarity 0.87)
• The Scaling Laws of Skills in LLM Agent Systems (NeurIPS 2026 · similarity 0.87)