Research map / Multi-agent systems and games / LLM agents
LLM multi-agent collaboration and safety
Studies collaboration, competition, evaluation, and monitoring of LLM-based multi-agent systems.
84 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2023–2026) · an approach in LLM agents, keyword cluster “llm agents · llms · agent systems”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
15 papers in 2023–24 and 69 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 23.1% to 6.0%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2023 · AdaPlanner: Adaptive Planning from Feedback with Language Models (NeurIPS 2023 · 12 citations)
- 2024 · ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · 55 citations)
- 2025 · OpenHands: An Open Platform for AI Software Developers as Generalist Agents (ICLR 2025 · 15 citations)
- 2026 · MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control (AAAI 2026 · 2 citations)
Earliest papers here
- AdaPlanner: Adaptive Planning from Feedback with Language Models (NeurIPS 2023 · 12 citations)
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · 55 citations)
- SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS 2024 · 30 citations)
What has been tried
42 papers here report 405 measured changes (a component added, removed or swapped, compared with a baseline): helped 69%, no clear effect 5%, hurt 23%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add diverse role prompts to the agents | FairEval · accuracy (%) | 53.8 -> 60 | Helped | ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · 55 citations) |
| Add linting guardrails to the edit action | SWE-bench Lite test · % Resolved | 15.0 -> 18.0 | Helped | SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (NeurIPS 2024 · 30 citations) |
| Delegate web browsing subtasks from CodeActAgent to specialized BrowsingAgent | WebArena · Success Rate (%) | 14.5 vs 14.8 for BrowsingAgent alone | No clear effect | OpenHands: An Open Platform for AI Software Developers as Generalist Agents (ICLR 2025 · 15 citations) |
| replace sliding-window history with cutoff | AgentBoard Avg. · progress rate | 41.4 -> 32.8 | Hurt | AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents (NeurIPS 2024 · 11 citations) |
| Use the Privacy-Enhancing Prompt instead of the Basic Prompt | PrivacyLens · leakage rate | GPT-4: 26.37 -> 24.54; Claude-3-Haiku: 38.95 -> 38.95; Mistral-7B-Instruct-v0.2: 33.47 -> 36.11 | No clear effect | PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action (NeurIPS 2024 · 10 citations) |
| add QA Engineer review | SWE-bench 25% subset · resolved ratio | 10.63 -> 13.94; 8.71 -> 10.63 | Helped | MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution (NeurIPS 2024 · 9 citations) |
| Add balancing after masking | BBQ · Bias Score | about 0.027 | Hurt | Mitigating Social Bias in Large Language Models: A Multi-Objective Approach Within a Multi-Agent Framework (AAAI 2025 · 7 citations) |
| Add safety requirements to the prompt | Toolemu benchmark · safety score | 2.007 -> 2.359 | Helped | Identifying the Risks of LM Agents with an LM-Emulated Sandbox (ICLR 2024 · 6 citations) |
| Use retrieval-augmented planning with plan decomposition and verification rather than retrieval-augmented planning alone | Downstream tasks · comprehensive score | Helped | AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML (ICML 2025 · 5 citations) | |
| use pyautogui code instead of JSON dict actions | Spider2-V task subset with a11ytree observations · success rate (%) | 10.5 -> 12.6 | Helped | Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? (NeurIPS 2024 · 5 citations) |
| Train on PR Mirror rather than LM Modify instances | SWE-bench Verified · % resolved | 5.7 (±1.5) -> 9.2 (±1.7) | Helped | SWE-smith: Scaling Data for Software Engineering Agents (NeurIPS 2025 · 4 citations) |
| Increase text history memory |H| from 3 to 9 or 15 | Border h=6, N=256 · cumulative p16 | 85.62 to 93.12 (|H|=3 to 9) | No clear effect | Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast (ICML 2024 · 4 citations) |
What reviewers will expect
Datasets and baselines named in the 69 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- SWE-bench Lite · 5 of 69 papers
- SWE-bench Verified · 4 of 69 papers
- WebArena · 4 of 69 papers
- HotpotQA · 3 of 69 papers
- MATH · 2 of 69 papers
- GSM8K · 2 of 69 papers
- MMLU · 2 of 69 papers
- AgentHarm · 2 of 69 papers
Baselines
- GPT-4o · 10 of 69 papers
- Claude-3.5-Sonnet · 8 of 69 papers
- Gemini-2.5-Pro · 8 of 69 papers
- GPT-4o-mini · 7 of 69 papers
- Claude-3.7-Sonnet · 6 of 69 papers
- AutoCodeRover · 4 of 69 papers
- Agentless · 4 of 69 papers
- GPT-5 · 4 of 69 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 84 papers; from OpenAlex and the papers' affiliations).
- Hong Kong University of Science and Technology · 5 papers
- Google · 5 papers
- Microsoft · 5 papers
- Carnegie Mellon University · 5 papers
- Zhejiang University · 4 papers
- Peking University · 4 papers
- Tsinghua University · 4 papers
- Stanford University · 4 papers
- Georgia Institute of Technology · 4 papers
- Shanghai Jiao Tong University · 4 papers
Use it
Download BibTeX of 21 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Agentic benchmarks and orchestration (185 papers)
- Software engineering agent benchmarks (129 papers)
- GUI agent memory and trajectories (72 papers)
Elsewhere in the topic
- LLM multi-agent decision making (194 papers)
- Self-evolving language agent learning (188 papers)
- Robust and efficient agent execution (174 papers)
- Agent data synthesis and behavior studies (108 papers)
- Agent safety and attack benchmarks (98 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).