Research map / Multi-agent systems and games / LLM agents
Scaling Large Language Model-based Multi-Agent Collaboration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MacNet, which arranges LLM agents in directed acyclic graphs (chain, tree, graph topologies) with critic-actor edge/node interactions, and studies how performance scales with agent count up to about a thousand agents.
Problem
Whether continually adding collaborating LLM agents improves task performance (analogous to neural scaling laws), and how inference-time agent topology affects scalability given context explosion and the limits of majority voting.
Approach
Organize agents as a DAG with a critic on each edge and an actor on each node, run dual-agent refinement along a topological order, and propagate only the final artifact (long-term memory control) so context grows linearly rather than quadratically with network size; then fit a sigmoid-variant "collaborative scaling law" as agents increase.
Key results
- MacNet-Random achieves the highest average quality of 0.6522 across the four tasks, versus 0.5805 for the best baseline (AgentVerse) and 0.5757 for CoT.
- Performance follows a logistic (sigmoid-variant) growth as agent count scales from 2^0 to 2^6 (over a thousand agents in a mesh), with collaborative emergence saturating at around a hundred agents.
- Random (irregular) topologies consume about 51.92% less time than mesh topologies while matching or exceeding their performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MacNet-Random instead of the strongest baseline | MMLU · MMLU | accuracy: 0.4485 -> 0.6877 | Helped |
| Use MacNet-Star instead of the strongest baseline | HumanEval · HumanEval | pass@k: 0.7256 -> 0.5549 | Hurt |
| Use MacNet-Chain instead of the strongest baseline | SRDD · SRDD | comprehensive metric: 0.7587 -> 0.8056 | Helped |
| Use MacNet-Tree instead of the strongest baseline | CommonGen-Hard · CommonGen-Hard | comprehensive metric: 0.6222 -> 0.7718 | Helped |
| Use MacNet-Random instead of the strongest baseline | All tasks · All tasks | Quality: 0.5805 -> 0.6522 | Helped |
| Remove agents' profiles (ablation) | MacNet · All topologies and tasks | average performance: 3.67% | Hurt |
| Use irregular random topology rather than regular topologies | MacNet · Across evaluated tasks | Quality | Helped |
| Increase interaction density from chain to tree to mesh | MacNet · Across evaluated tasks | average performance | Helped |
| Use a wider star rather than a deeper tree at the same density | MacNet · Across evaluated tasks | performance | Helped |
| Reverse the directions of asymmetric topologies | MacNet · Reversed topologies | performance | Hurt |
| Scale the network by increasing the number of agents | MacNet · Across topologies | Quality | Helped |
| Scale majority-voting baselines by adding agents | Majority voting, including CoT or AutoGPT · Closed-domain tasks | performance: +0.9% | No clear effect |
Surprising findings
- Irregular random topologies outperform denser regular topologies such as mesh, which the authors attribute to information overload in dense networks.
- Reversing edge directions of topologies causes significant performance degradation, with divergent topologies beating their convergent counterparts.
Limitations
- Experiments primarily use GPT-3.5 with a fixed set of four benchmarks; open-source model results are only mentioned in a footnote.
- The fitted sigmoid scaling law is a general pattern over network scale and does not account for profiles, tools, or communication protocols.
- Artifact version rollbacks occur in deep topologies due to memory control, causing loss of distant agents' context.
Experimental setup
Datasets: MMLU HumanEval SRDD CommonGen-Hard
Benchmarks: MMLU HumanEval SRDD CommonGen-Hard
Metrics: accuracy pass@k quality score (completeness, executability, consistency; gra statistical significance (p<=0.05)
Baselines: CoT AutoGPT GPTSwarm AgentVerse
Builds on: chain-of-thought AgentVerse GPTSwarm AutoGPT graph-of-thought
Closest papers by meaning
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors (ICLR 2024 · similarity 0.90)
- Emergent Coordination in Multi-Agent Language Models (ICLR 2026 · similarity 0.89)
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.88)
- Multi-Agent Collaboration via Evolving Orchestration (NeurIPS 2025 · similarity 0.88)
- G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (NeurIPS 2025 · similarity 0.87)
- Autonomous Agents for Collaborative Task under Information Asymmetry (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.