Research map / Multi-agent systems and games / LLM agents

EvoMAS: Evolutionary Generation of Multi-Agent Systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EvoMAS, which evolves structured MAS configurations (topology, prompts, models, tools) via LLM-guided mutation and crossover with execution feedback, a pool, and experience memory, outperforming human-designed and automatic MAS generation baselines on BBEH, WorkBench, and SWE-Bench.

Problem

Automatic generation of LLM-based multi-agent systems is brittle (code generation has executability failures) or constrained by rigid templates, and existing methods do not jointly optimize the whole MAS or accumulate experience across tasks.

Approach

Represent a MAS as a declarative configuration (communication graph, per-agent backbone, prompt, tool set) and evolve a pool of such configurations with LLM-driven mutation and crossover conditioned on execution traces, with consolidated evolution memory reused across queries.

Key results

• EvoMAS with LLM selection reaches 58.7% on BBEH, 10.5 points above the best EvoAgent result (48.2%).
• EvoMAS reaches 63.8% on SWE-Bench-Verified with Claude/Qwen backbones, and 79.1% with Claude-4.5-Sonnet, versus 78.8% for the top public leaderboard entry as reported by the authors.
• Execution rate of EvoMAS is 98.9% on BBEH and 98.4% on SWE-Bench-Verified, while MAS-GPT reaches only 71.5% on BBEH and 1.2% on SWE-Bench-Lite.

Surprising findings

• Qwen3-235B outperforms the larger Qwen3-480B on BBEH (52.8% vs 43.0% for per-LLM EvoMAS).
• Scaling a single agent via looping to matched token budget yields no gain on SWE-Bench-Verified.

Limitations

• Evolution incurs substantial token and time overhead before the pool stabilizes (~300-400 queries).
• Only cooperative MAS are considered; adversarial coordination is not studied.
• Relies on LLM-as-judge reward as proxy, with a gap to oracle reward of 1.5-3.4 points.

Experimental setup

Datasets: BBEH BBEH-Mini WorkBench SWE-Bench-Lite SWE-Bench-Verified AIME 2024-2025

Benchmarks: BBEH WorkBench SWE-Bench-Lite SWE-Bench-Verified AIME 2024-2025

Metrics: accuracy resolved rate execution rate

Baselines: Direct LLM Call Single Agent MetaGPT ChatDev Peer Review Croto SMoA Multi-Agent Debate Majority Vote AutoAgents MAS-GPT ADAS

Builds on: EvoAgent ADAS MAS-GPT AutoAgents smolagents

Closest papers by meaning

• MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.90)
• CATArena: Evaluating Evolutionary Capabilities of Code Agents via Iterative Tournaments (ICML 2026 · similarity 0.89)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.88)
• Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.88)
• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.88)
• Automated Design of Agentic Systems (ICLR 2025 · similarity 0.88)