Research map / Multi-agent systems and games / LLM agents

The Illusion of Multi-Agent Advantage

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically re-evaluates automatically generated multi-agent systems against a budget-matched CoT self-consistency baseline, introduces a synthetic multi-hop financial benchmark (SMFR) with an expert-designed MAS, and dissects generated workflows for architectural bloat.

Problem

Whether automatically generated multi-agent systems (MAS) actually outperform single-agent systems (SAS) such as CoT with self-consistency once compute cost is controlled, and whether their claimed MAS mechanisms (decomposition, parallelism, context separation) actually manifest in practice.

Approach

Compares six automatic MAS frameworks against CoT-SC across reasoning, coding and browsing benchmarks with cost accounting, adds a diagnostic synthetic benchmark (SMFR) where decomposition and parallelism are explicit, and deconstructs generated workflows to find role redundancy, positional bias, and collapse into CoT-SC-like ensembles.

Key results

• On standard benchmarks CoT-SC generally matches or beats automatic MAS frameworks at a fraction of cost; e.g., GPT-5 CoT-SC reaches 87.35% on GPQA-Diamond at $46 versus MAS-Zero 86.75% at $637.
• On SMFR, Expert-MAS raises GPT-5 from 56.97% (CoT-SC) to 96.51% and GPT-OSS-120B from 26.13% to 36.14%, while automatic MAS rarely surpass CoT-SC and never economically.
• Architectural analysis finds MaAS collapses to a single I/O call on 74.2% of BrowseComp-Plus activations and about 70% of GPT-4o debate cases reach unanimous consensus.

Surprising findings

• A single-agent GPT-5 CoT-SC outperforms the most sophisticated GPT-4o-based MAS frameworks while using less than half the tokens.
• An all-assistant DyLAN configuration (54.4%) matched or beat task-specific expert roles (53.4%).

Limitations

• Automatic MAS evaluated with reduced search iterations for GPT-5 due to cost, and Gemini-2.5-Pro results use a single run.
• GPT-OSS-120B excluded from SWE-Bench Lite; MAS-Orchestra and Gemini runs incomplete in several settings.
• SMFR is a single synthetic financial task, and Expert-MAS is hand-designed for it, so the diagnostic may not generalize.

Experimental setup

Datasets: GPQA-Diamond HLE-Maths SWE-Bench Lite BrowseComp-Plus SMFR

Benchmarks: GPQA-Diamond HLE-Maths SWE-Bench Lite BrowseComp-Plus SMFR

Metrics: accuracy cost (USD)

Baselines: CoT CoT-SC DyLAN MAS-Zero ADAS AFlow MaAS MAS-Orchestra

Builds on: DyLAN MAS-Zero ADAS AFlow MaAS MAS-Orchestra

Closest papers by meaning

• MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks (ICML 2026 · similarity 0.90)
• MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems (ICML 2026 · similarity 0.88)
• Benefits and Limitations of Communication in Multi-Agent Reasoning (ICLR 2026 · similarity 0.87)
• Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.87)
• MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.87)
• EvoMAS: Evolutionary Generation of Multi-Agent Systems (ICML 2026 · similarity 0.87)