Research map / Multi-agent systems and games / LLM agents

Building Cooperative Embodied Agents Modularly with Large Language Models

ICLR 2024 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 39 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CoELA, a cognitive-inspired modular LLM agent (perception, memory, communication, planning, execution) for decentralized, costly-communication multi-agent embodied cooperation, evaluated on TDW-MAT and C-WAH with GPT-4, LLAMA-2, and a LoRA-tuned CoLLAMA plus a human user study.

Problem

Multi-agent embodied cooperation under decentralized control, raw sensory observations, costly natural-language communication, and long-horizon multi-objective household tasks, where it is unclear whether LLMs can plan and communicate well.

Approach

A cognitive-architecture-inspired modular framework where LLMs act as the communication (what to say) and planning (what to do, including when to communicate) modules, with separate perception, semantic/episodic/procedural memory, and a rule/A*-based execution module that turns high-level plans into primitive actions.

Key results

• CoELA with GPT-4 reaches 0.71 total Transport Rate on TDW-MAT (+39% efficiency improvement) vs 0.43 for RHP alone, and 0.69 when paired with RHP+CoELA.
• On C-WAH with symbolic observation, CoELA+CoELA needs 57 average steps vs 111 for MHP alone (49% efficiency improvement).
• In a human study (8 subjects, 80 trials), trust score was 6.3 for CoELA vs 4.7 for CoELA without communication (p=0.0003).

Measured effects reported in the paper

Surprising findings

• Open LLAMA-2-13b-chat substantially underperforms GPT-4 as the driver, but LoRA fine-tuning on only 572 filtered trajectories yields CoLLAMA-2 with 0.70 TR, close to GPT-4's 0.71.
• Disabling communication among AI agents caused no significant performance drop on C-WAH, indicating that effective LLM-driven communication is still hard in this setting.

Limitations

• Limited use of 3D spatial information since pure text LLMs are used.
• Planner operates on abstract high-level plans and cannot reason over low-level actions, leading to ineffective decisions.
• Unstable complex reasoning such as miscounting remaining goals; results mostly single runs for CoELA due to cost.

Experimental setup

Datasets: TDW-MAT C-WAH TDW-House MS COCO

Benchmarks: TDW-MAT C-WAH

Metrics: Transport Rate Average Steps Efficiency Improvement trust rating (7-point Likert)

Baselines: MCTS-based Hierarchical Planner (MHP) Rule-based Hierarchical Planner (RHP) Multi-Agent Transformer (MAT) GPT-3.5 LLAMA-2-13b-chat

Builds on: Watch-And-Help ThreeDWorld Transport Challenge Multi-Agent Transformer chain-of-thought prompting LoRA

Closest papers by meaning

• CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation (ICLR 2025 · similarity 0.90)
• LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior (ICML 2026 · similarity 0.90)
• Collaborative Tree Search for Enhancing Embodied Multi-Agent Collaboration (CVPR 2025 · similarity 0.89)
• COOP$^2$: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems (NeurIPS 2026 · similarity 0.89)
• More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration (ICML 2026 · similarity 0.89)
• REVECA: Adaptive Planning and Trajectory-Based Validation in Cooperative Language Agents Using Information Relevance and Relative Proximity (AAAI 2025 · similarity 0.88)