Research map / Multi-agent systems and games / LLM agents
Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Hypothetical Minds, an LLM-based embodied agent whose Theory of Mind module generates, evaluates (via Rescorla-Wagner value updates) and refines natural-language hypotheses about other agents, evaluated on Melting Pot scenarios.
Problem
MARL methods struggle with non-stationarity, sample efficiency, and generalization to novel agents in multi-agent social environments; LLM agents have mainly been limited to collaborative settings.
Approach
A cognitively inspired LLM agent whose ToM module generates natural-language hypotheses about other agents' strategies, scores them by predictive accuracy using a Rescorla-Wagner value update with intrinsic reward, refines top-k hypotheses, and conditions high-level planning on validated hypotheses.
Key results
- HM achieves the highest reward on 9/9 RWS Repeated scenarios (e.g., 50.8 on scenario 0 vs. 21.0 for Reflexion and 0.0 for OPRE).
- HM is best on 6/8 RWS Arena scenarios and on all 3 Collaborative Cooking Asymmetric partner scenarios (e.g., 628.3 vs. 292.6 for Reflexion on skilled partner).
- On Prisoner's Dilemma, HM is the best LLM agent in 5/10 scenarios, but OPRE achieves the highest reward overall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace single-call Vanilla Mind Prompting with separate calls for reasoning steps in Modular Mind Prompting (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Helped |
| Add hypothesis evaluation to Modular Mind Prompting (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Helped |
| Add hypothesis refinement to MMP + Hypothesis Evaluation (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Helped |
| Remove ToM-specific prompting while retaining hypothesis evaluation and refinement (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Mixed |
| Retain hypotheses beyond the last generated one by using top_k greater than 0 (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Helped |
| Increase top_k beyond 3 (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | No clear effect |
| Increase top_k past 5 (ablation) | Hypothetical Minds · Running With Scissors Repeated | average reward | Hurt |
| Vary the hypothesis-value learning rate α (ablation) | Hypothetical Minds with Llama-3-70B-Instruct · Running With Scissors Repeated | average reward | No clear effect |
| Vary the hypothesis validation threshold (ablation) | Hypothetical Minds with Llama-3-70B-Instruct · Running With Scissors Repeated | average reward | No clear effect |
| Replace GPT-4 with GPT-3.5 as the base LLM (ablation) | Hypothetical Minds · Melting Pot environments | average reward | Hurt |
| Replace GPT-4 with Llama-3-70B-Instruct as the base LLM (ablation) | Hypothetical Minds · Melting Pot environments | average reward | Mixed |
Surprising findings
- OPRE, a MARL method designed for an earlier RWS version, gets near-zero reward on nearly every RWS Repeated scenario.
- Higher ToM costs (>$10 per episode) correlate with lower performance because successful hypothesis validation reduces subsequent LLM calls.
Limitations
- Requires human-designed scaffolding, prompts, and encoded game rules rather than learning them from the environment.
- LLM agents lack low-level spatial control; HM travels inefficiently to collect resources, losing to OPRE on Prisoner's Dilemma.
- Llama-3 could not be evaluated on RWS Arena due to context length limits.
Experimental setup
Datasets: Melting Pot 2.0 substrates (Running With Scissors Repeated, Running With Scissors Arena, Collaborative Cooking Asymmetri
Benchmarks: Melting Pot 2.0 RWS Repeated Melting Pot RWS Arena Melting Pot Collaborative Cooking Asymmetric Melting Pot Prisoner's Dilemma in the Matrix Repeated
Metrics: average reward per episode SEM
Baselines: ReAct Reflexion PlanReAct PPO OPRE Vanilla Mind Prompting Modular Mind Prompting
Builds on: Generative Agents ReAct Reflexion Tree-of-Thoughts OPRE
Closest papers by meaning
- Mental-Models for Multi-Agent Systems (NeurIPS 2026 · similarity 0.91)
- Adaptive Theory of Mind for LLM-based Multi-Agent Coordination (AAAI 2026 · similarity 0.89)
- ReMA: Learning to Meta-Think for LLMs with Multi-agent Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
- Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.88)
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.88)
- AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.