Research map / Multi-agent systems and games / LLM agents

Large Language Models Miss the Multi-agent Mark

NeurIPS 2025 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that multi-agent LLM (MAS LLM) systems ignore core multi-agent systems principles (social agency, environment design, asynchronous coordination and standardized communication, and quantifiable emergence) and proposes research directions to address these gaps.

Problem

Many frameworks labelled as multi-agent LLM systems appropriate MAS terminology without engaging with MAS theory, leading to LLM-centric architectures lacking autonomy, social interaction, structured environments, asynchrony, and rigorous measures of emergent behaviour.

Approach

Systematically contrasts MAS theory (social reactivity/proactivity, environment dimensions, concurrent/asynchronous systems, performative communication such as KQML, weak/strong emergence) with current MAS LLM practice, and advocates pretraining for cooperation, non-LLM-centric environments, native asynchrony with standard protocols, and falsifiable definitions of emergence.

Key results

• Literature survey of roughly 110-112 MAS LLM Benchmark & Evaluation papers (2023-2025) finds most operate in partially observable, deterministic, temporally dynamic, discrete, textual environments.
• Of more than 1400 MAS LLM articles, only 22 explicitly model or discuss asynchronous agent interactions.
• Of more than 60 papers on emergent behaviour, only a few define clear metrics to quantify emergence while most evaluate qualitatively.

Surprising findings

• A recent survey reports 37% of MAS LLM failure cases stem from inter-agent misalignment or coordination issues.
• Setting LLM temperature to zero is argued to not guarantee deterministic behaviour in MAS LLM environments.

Limitations

• Literature analysis relies on a curated list of papers (e.g., ~112 from a GitHub repository), not a systematic review.
• Argumentative position without new empirical experiments.
• Counter-arguments (social pretraining unnecessary, orchestration sufficient) are acknowledged but only briefly addressed.

Experimental setup

Builds on: ReAct Chain of Thought Tree of Thought AutoGen KQML

Closest papers by meaning

• Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 · similarity 0.90)
• MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems (ICLR 2026 · similarity 0.90)
• AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents (NeurIPS 2024 · similarity 0.89)
• More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration (ICML 2026 · similarity 0.89)
• MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systems (ICML 2025 · similarity 0.89)
• Emergent Coordination in Multi-Agent Language Models (ICLR 2026 · similarity 0.88)