Research map / Multi-agent systems and games / Human-AI and goal-conditioned
MindForge: Empowering Embodied Agents with Theory of Mind for Lifelong Cultural Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MindForge (called CollabVoyager in the text), a Voyager extension where LLM agents use structured BDI/BigToM theory-of-mind representations, natural-language in-game chat, and multi-part memory to learn from expert or matched peers in Minecraft without weight updates.
Problem
Voyager-style lifelong learning agents built on open-weight LLMs fail even on elementary Minecraft tasks, and they learn in isolation without social learning.
Approach
Combines a BigToM causal template (percepts, beliefs, desires, actions) with partner-specific belief models and natural-language chat between agents, plus episodic, semantic and procedural memory, so weak open-weight agents improve through perspective-taking communication rather than fine-tuning.
Key results
- Mistral-7B with MindForge reaches 37.5% on dirt collection versus 7% for plain Mistral-7B Voyager, and GPT-4 scores 100%.
- In the collaborative setting, Mixtral-8x7B population success rises from 62% to 79% after seven communication turns with a GPT-4 seed round.
- Llama 3.1-70B MindForge reaches the wooden and stone tool milestones where open-weight Voyager reaches only wooden tools; the abstract reports 3x more tech-tree milestones and 2.3x more unique items.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MindForge collaboration to Voyager | Mistral-7B agent · Dirt collection | task completion rate: 7% -> 37.5% | Helped |
| Add MindForge collaboration to Voyager | Mistral-7B agent · Wood collection | task completion rate: 4% -> 33.3% | Helped |
| Apply PEFT fine-tuning using execution traces and Minecraft documentation | Mistral-7B Voyager · Dirt collection | task completion rate: 7% -> 7% | No clear effect |
| Apply PEFT fine-tuning using execution traces and Minecraft documentation | Mistral-7B Voyager · Wood collection | task completion rate: 4% -> 12.5% | Helped |
| Add GPT-4 expert communication rounds | Llama 3.1-70B MindForge · Mine Iron | task completion rate: 41% -> 62% | Helped |
| Add GPT-4 expert communication rounds | Llama 3.1-70B MindForge · Craft a Pickaxe | task completion rate: 20% -> 45% | Helped |
| Add matched-capability peer communication rounds | Llama 3.1-70B MindForge · Craft a Pickaxe | task completion rate: 20% -> 33% | Helped |
| Add perspective-taking during communication (ablation) | MindForge · Dirt collection | task completion rate: 54% -> 67% | Helped |
| Use structured ToM instead of unstructured perspective-taking (ablation) | MindForge · Craft a Pickaxe | task completion rate: 41% -> 45% | Helped |
| Use structured ToM instead of unstructured perspective-taking (ablation) | MindForge · Mine Iron | task completion rate: 58% -> 62% | Helped |
| Retain episodic memory (ablation) | MindForge with Mixtral-8x7B · Dirt collection | task completion rate: 25% -> 29.15% | Helped |
| Retain episodic memory (ablation) | MindForge with Mixtral-8x7B · Wood collection | task completion rate: 45.83% -> 50% | Helped |
Surprising findings
- PEFT fine-tuning of Mistral-7B on GPT-4 execution traces and Minecraft manuals gave no benefit on dirt collection (7%).
- In the collaborative setting, purely peer-to-peer collaboration between unprimed weak agents often stagnates or degrades performance.
Limitations
- No standalone limitations section; the paper's checklist states limitations are discussed throughout.
- Experiments are limited to Minecraft tasks and small trial counts (e.g., 24 trials per task).
- Weak agents like Mistral-7B rarely seek help even when a flexible communication protocol is offered.
Experimental setup
Datasets: Voyager execution logs generated by GPT-4 Minecraft Wiki Minecraft API documentation
Benchmarks: Minecraft tech-tree (wooden, stone, iron tools) Minecraft dirt/wood collection tasks Mine iron Craft a pickaxe
Metrics: task completion rate prompting iterations to milestone unique items collected milestones reached
Baselines: Voyager ReAct Reflexion AutoGPT PEFT fine-tuned Mistral-7B
Builds on: Voyager BigToM Soar Belief-Desire-Intention Mineflayer
Closest papers by meaning
- Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models (ICLR 2025 · similarity 0.88)
- ADAM: An Embodied Causal Agent in Open-World Environments (ICLR 2025 · similarity 0.87)
- CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter (NeurIPS 2026 · similarity 0.87)
- Thought Communication in Multiagent Collaboration (NeurIPS 2025 · similarity 0.87)
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.87)
- AgentOdyssey: Open-Ended Long-Horizon Text Game Generation for Test-Time Continual Learning Agents (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.