Research map / Multi-agent systems and games / LLM agents
Multi-Agent Collaboration via Evolving Orchestration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Puppeteer, a centralized RL-trained orchestrator that sequentially activates LLM agents in a multi-agent system, evolving compact cyclic reasoning structures with better accuracy and lower token cost.
Problem
Static or predefined agent topologies in LLM multi-agent systems incur coordination overhead and inefficiency as task complexity and agent count grow.
Approach
A centralized learnable policy (puppeteer) selects which agent to activate at each step given the evolving task state, serializing collaboration into a topological sequence, and is trained with REINFORCE using a reward that penalizes token/FLOP cost via a log step-cost term.
Key results
- In the Titan subspace, Puppeteer's average score rises from 0.6893 (initialized) to 0.7731 (evolved), above the best baseline AFlow with Llama-3.1-405B at 0.6899.
- In the Mimas subspace, Puppeteer's evolved average is 0.6324 versus 0.5364 for AFlow with Llama-3.1-8B.
- Token consumption decreases over training in most settings, and the number of agents drops in the Titan setting, indicating compaction and cyclic structures emerge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evolve the Puppeteer policy through online reinforcement learning | Puppeteer in Titan · Average across GSM-Hard, MMLU-Pro, SRDD, and CommonGen-Hard | average score: 0.6893 -> 0.7731 | Helped |
| Evolve the Puppeteer policy through online reinforcement learning | Puppeteer in Mimas · Average across GSM-Hard, MMLU-Pro, SRDD, and CommonGen-Hard | average score: 0.6273 -> 0.6324 | Helped |
| Evolve the Puppeteer policy | Puppeteer in Mimas · GSM-Hard | accuracy: 0.5600 -> 0.5400 | Hurt |
| Evolve the Puppeteer policy | Puppeteer in Mimas · SRDD | score: 0.6653 -> 0.6266 | Hurt |
| Evolve the Puppeteer policy | Puppeteer in Titan · CommonGen-Hard | score: 0.7381 -> 0.7987 | Hurt |
| Use the W4D2 default instead of increasing exploration width or depth (ablation) | Puppeteer · Reasoning tasks (settings shown in Figure) | accuracy | Mixed |
| Optimize the orchestrator with the reward balancing solution quality and token cost | Puppeteer · Tasks across Titan and Mimas | token consumption | Helped |
| Increase structural compaction and cyclic reasoning through orchestration evolution | Puppeteer · Multi-agent reasoning topologies | graph density and cycle formation | Helped |
Surprising findings
- The Mimas setting keeps a stable number of orchestrated agents, with token savings coming from choosing cheaper agents rather than shorter chains.
- MacNet with Llama-3.1-405B achieves the highest CommonGen-Hard score (0.8817) yet performs poorly on other tasks.
Limitations
- Reward relies on coarse final-output and token signals without step-level supervision.
- Assumes a fixed set of agents and tools that cannot be modified at inference.
- Occasional mis-coordination or deceptive agreement among agents.
Experimental setup
Datasets: GSM-Hard MMLU-Pro SRDD CommonGen-Hard ALFWorld
Benchmarks: GSM-Hard MMLU-Pro SRDD CommonGen-Hard
Metrics: accuracy SRDD official score (completeness, executability, consistenc CommonGen-Hard aggregate score token consumption number of agents
Baselines: Self-Refine AFlow MacNet EvoAgent Llama-3.1-405B GPT-4-Turbo GPT-4o-Mini Llama-3.1-8B Mistral-Nemo-12B Qwen-2.5-14B
Builds on: REINFORCE MacNet EvoAgent AFlow Self-Refine
Closest papers by meaning
- Reward Modeling for Multi-Agent Orchestration (NeurIPS 2026 · similarity 0.90)
- AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration (ICML 2026 · similarity 0.89)
- AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systems (NeurIPS 2025 · similarity 0.89)
- Emergent Coordination in Multi-Agent Language Models (ICLR 2026 · similarity 0.89)
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.89)
- Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.