Research map / Multi-agent systems and games / LLM agents
MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MetaAgent, which automatically designs finite-state-machine (FSM) based multi-agent systems from a task description, merges redundant states with an LLM adaptor, and deploys them with tool use and state traceback.
Problem
Existing human-designed multi-agent systems cover few scenarios, and automated design methods lack tool integration, depend on external training data, and use rigid communication structures without traceback.
Approach
An LLM designer produces an FSM whose states each have an assigned agent, instruction, listeners, and a per-state condition verifier that can stay in state (null-transition), advance, or trace back; an LLM-driven pairwise merging step removes redundant states without external data. Linear, debate, and orchestrator structures are presented as special cases of this FSM.
Key results
- MetaAgent scores 0.86 on Trivial Creative Writing and 0.60 on GPQA Diamond, above the best prompt baseline (Self-Refine 0.76 and 0.55).
- On ML-Bench, MetaAgent averages 0.83 normalized performance, versus 0.86 for the human-designed Data Interpreter and 0.16 for SPP.
- On five software development tasks, MetaAgent averages 0.85 checkpoint pass ratio, versus 0.35 for MetaGPT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MetaAgent instead of SPP | GPT-4o multi-agent system · Trivial Creative Writing | success rate: 0.79 -> 0.86 | Helped |
| Use MetaAgent instead of Self-Refine | GPT-4o system · GPQA (Diamond) | success rate: 0.55 -> 0.60 | Helped |
| Use MetaAgent instead of Data Interpreter | ML_Bench system · ML_Bench average | normalized performance score: 0.86 -> 0.83 | Hurt |
| Use MetaAgent instead of MetaGPT | software development system · Software development average | passed-checkpoint ratio: 0.35 -> 0.85 | Helped |
| Remove FSM state optimization (ablation) | MetaAgent · ML_Bench average | normalized performance score: 0.83 -> 0.61 | Hurt |
| Remove FSM state optimization (ablation) | MetaAgent · Software development average | passed-checkpoint ratio: 0.85 -> 0.65 | Hurt |
| Remove FSM state optimization (ablation) | MetaAgent · Trivial Creative Writing | success rate: 0.86 -> 0.65 | Hurt |
| Remove FSM state optimization (ablation) | MetaAgent · GPQA (Diamond) | success rate: 0.60 -> 0.56 | Hurt |
| Disable tool use (ablation) | MetaAgent · Trivial Creative Writing | success rate: 0.86 -> 0.79 | Hurt |
| Disable tool use (ablation) | MetaAgent · GPQA (Diamond) | success rate: 0.60 -> 0.52 | Hurt |
| Remove state traceback (ablation) | MetaAgent · ML_Bench average | normalized performance score: 0.83 -> 0.72 | Hurt |
| Remove state traceback (ablation) | MetaAgent · Software development average | passed-checkpoint ratio: 0.85 -> 0.35 | Hurt |
Surprising findings
- Downgrading the executor from GPT-4o to GPT-3.5-Turbo hurt ML-Bench performance more (0.35) than downgrading the designer (0.26) when both were swapped, which the authors read as executor quality mattering more.
- AutoAgents scores 0.00 on ML-Bench, indicating that auto-designed baselines without tool feedback can fail completely on these tasks.
Limitations
- Results rely on GPT-4o with temperature 0 and a small number of tasks (5 ML tasks, 5 software tasks), so variance is not reported.
- Merging and FSM design depend on the LLM designer and adaptor, and no open-weight models were evaluated.
Experimental setup
Datasets: Trivial Creative Writing GPQA Diamond ML-Bench
Benchmarks: Trivial Creative Writing GPQA ML-Bench Software development tasks (2048, Snake, Brick breaker, Excel, Weather)
Metrics: success rate normalized performance score checkpoint pass ratio token cost
Baselines: Direct prompting CoT CoT-SC llm-debate Self-Refine SPP AutoAgents MetaGPT
Builds on: Solo Performance Prompting AutoAgents Automated Design of Agentic Systems MetaGPT Data Interpreter
Closest papers by meaning
- Automated Design of Agentic Systems (ICLR 2025 · similarity 0.89)
- MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.89)
- MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework (ICLR 2024 · similarity 0.89)
- Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.89)
- AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.88)
- AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.