Research map / Multi-agent systems and games / LLM agents

Mental-Models for Multi-Agent Systems

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes mental-model-enabled agents that learn an amortized recursive Theory-of-Mind latent (first- and second-order belief/intent/thought) jointly with a belief-conditioned reward model, then train a standalone policy with mental-reward-guided GRPO.

Problem

LLM agents in multi-agent, partially observable interactions do not maintain an explicit, reusable latent model of a partner's hidden goals, beliefs, and strategies; prompt- or summary-based partner modeling is limited in expressivity and tightly coupled to prompt design.

Approach

An amortized recursive latent (VAE-style Gaussian posteriors, with z2 conditioned on z1, split into belief/intent/thought subspaces) is trained jointly with a utility head that scores candidate actions conditioned on the latent; the frozen scorer's reward then drives GRPO so that the deployed policy internalizes partner-aware action selection without needing the latent at inference.

Key results

• On Sotopia-all with GPT-4o-mini partner, Qwen2.5-7B overall score rises from 3.290 to 3.856 with the method.
• On BigToM with initial belief, the method reaches TB∧FB 97.5 vs base 48.0 (paired accuracy) and beats GPT-4 (CoT) on several splits.
• On MMRole, the method scores 1.023 overall vs 0.994 for MMRole-Agent, with 1.019 vs 0.981 on the Out split.

Surprising findings

• Scrambling the latent of a flat compression VAE barely affects reward prediction, while scrambling the mental-model latent sharply disrupts it.
• On ToMi, the method improves over base but classic task-specific ToMi baselines (e.g., EntNet at 90.60 average) remain stronger.

Limitations

• Training environments are primarily dyadic; multi-party evidence is only zero-shot FANToM transfer.
• Evaluated mainly on 7B-scale backbones; scaling to larger policies untested.
• No human-in-the-loop evaluation; relies on GPT-4o-generated augmentation annotations.

Experimental setup

Datasets: Sotopia-pi BigToM ToMi MMRole Craigslist-bargain FANToM

Benchmarks: Sotopia BigToM ToMi MMRole Craigslist-bargain

Metrics: Sotopia 7-dimension overall score TB/FB/TB∧FB accuracy Average and Joint accuracy MMRole normalized role-play score Craigslist deal/walkaway/no-agreement rate

Baselines: GPT-4o-mini Gemini SFT GRPO MMRole-Agent GPT-4 ToMi classic baselines (MemNN, RelNet, EntNet) GPT-4o

Builds on: GRPO ToMnet Sotopia-pi BigToM MMRole

Closest papers by meaning

• Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models (ICLR 2025 · similarity 0.91)
• AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.90)
• Partner Modelling Emerges in Recurrent Agents (But Only When It Matters) (NeurIPS 2025 · similarity 0.89)
• Thought Communication in Multiagent Collaboration (NeurIPS 2025 · similarity 0.89)
• Adaptive Theory of Mind for LLM-based Multi-Agent Coordination (AAAI 2026 · similarity 0.89)
• G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (NeurIPS 2025 · similarity 0.89)