Research map / Multi-agent systems and games / LLM agents
Multi-agent Architecture Search via Agentic Supernet
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MaAS, which optimizes an agentic supernet (a probabilistic distribution over multi-agent architectures) and samples query-dependent multi-agent systems to balance accuracy and inference cost.
Problem
Automated multi-agent system design typically searches for a single static workflow, which cannot adapt resource allocation (LLM calls, tokens, cost) to the difficulty and domain of each query.
Approach
Replace the search for one optimal agentic workflow with an agentic supernet: a layered probability distribution over agentic operators, sampled by a controller network per query, with an early-exit operator for query-dependent depth. Distribution parameters are updated by Monte Carlo gradient estimates with cost-aware weights, and operators are updated via LLM-generated textual gradients.
Key results
- MaAS reaches 83.59% average across GSM8K, MATH, MultiArith, HumanEval, and MBPP with gpt-4o-mini, versus 82.25% for the best baseline AFlow.
- On MATH, MaAS costs $0.42 per inference vs $1.66 for AFlow, and its optimization costs $3.38 vs $22.50 for AFlow.
- On GAIA, MaAS averages 20.69% vs 16.34% for the best automated baseline AgentSquare; the paper reports up to 18.38% gains on Level 1 over vanilla LLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MaAS instead of AFlow | multi-agent architecture search · GSM8K | accuracy: 91.16 -> 92.30 | Helped |
| use MaAS instead of AFlow | multi-agent architecture search · MATH | accuracy: 51.28 -> 51.82 | Helped |
| use MaAS instead of AgentSquare | multi-agent architecture search · MultiArith | accuracy: 97.77 -> 98.80 | Helped |
| use MaAS instead of AFlow | multi-agent architecture search · HumanEval | pass@1: 90.93 -> 92.85 | Helped |
| use MaAS instead of AFlow | multi-agent architecture search · MBPP | pass@1: 81.67 -> 82.17 | Helped |
| use MaAS instead of TapeAgent | multi-agent architecture search · GAIA Avg. | accuracy: 16.61 -> 20.69 | Helped |
| add textual gradient (ablation) | MaAS · HumanEval | pass@1: 90.17 -> 92.85 | Helped |
| add textual gradient (ablation) | MaAS · MATH | accuracy: 48.23 -> 51.82 | Helped |
| add early-exit operator (ablation) | MaAS · HumanEval | pass@1: 91.44 -> 92.85 | Helped |
| add early-exit operator (ablation) | MaAS · MATH | accuracy: 51.53 -> 51.82 | No clear effect |
| add cost constraint (ablation) | MaAS · HumanEval | pass@1: 92.94 -> 92.85 | No clear effect |
| add cost constraint (ablation) | MaAS · MATH | accuracy: 51.19 -> 51.82 | Helped |
Surprising findings
- Removing the textual-gradient operator update causes the largest performance drop, while removing the early-exit operator or cost constraint has little effect on accuracy.
- Operators are sampled from the supernet by query, and the Debate operator, held out during training, is still selected reasonably at inference (inductive generalization).
Limitations
- Experiments rely on closed-source gpt-4o-mini via API, and the cost metrics are tied to API pricing.
- Operator set is hand-initialized (CoT, Debate, ReAct, etc.), so the search is bounded by the predefined operators.
- Benchmark subsets are small (e.g., 33 training problems for HumanEval), and results on some benchmarks are single-run with no variance reported.
Experimental setup
Datasets: GSM8K MATH MultiArith HumanEval MBPP GAIA
Benchmarks: GSM8K MATH MultiArith HumanEval MBPP GAIA
Metrics: accuracy pass@1 prompt/completion tokens API cost ($) wall-clock time
Baselines: CoT ComplexCoT Self-Consistency MultiPersona LLM-Debate LLM-Blender DyLAN AgentVerse MacNet AutoAgents GPTSwarm ADAS
Builds on: AFlow ADAS AgentSquare GPTSwarm LLM-Debate
Closest papers by meaning
- Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems (ICML 2026 · similarity 0.89)
- AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.88)
- Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.88)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.88)
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.87)
- MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.