Research map / Large language models / Agents, code and math
Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ExploreToM, an A*-guided, DSL-based generator of adversarial theory-of-mind stories that yields very hard evaluation sets and training data that improves Llama-3.1 8B on ToMi.
Problem
Existing theory-of-mind benchmarks are limited in scenario diversity, are often saturated or leaked, and may overestimate LLM ToM ability; training data that explicitly requires ToM is scarce.
Approach
Searches a domain-specific language of ToM story actions with A* (using target-model accuracy on automatically generated questions as cost) to find story structures that fool the model, then infills them into natural text with an LLM, with asymmetric (secret witness) belief updates added.
Key results
- ExploreToM-generated story structures reduce GPT-4o accuracy to as low as 9% and Llama-3.1 70B to 0% on first-order questions.
- Fine-tuning Llama-3.1 8B on ExploreToM data (ExploreToM-8B) raises ToMi accuracy from 68% to 95% (+27) and Hi-ToM from 30% to 59%.
- A* search yields a dataset about 2 accuracy points more challenging than over-generation and filtering under the same budget.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · ToMi | accuracy: 68% -> 95% (+27%) | Helped |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · Hi-ToM | accuracy: 30% -> 59% (+29%) | Helped |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · BigToM | accuracy: 75% -> 81% (+6%) | Helped |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · OpenToM | F1: .39 -> .46 (+.07) | Helped |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · FANToM | accuracy: 0.3% -> 0.2% (-0.1%) | Hurt |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · Multi3Woz | accuracy: 96% -> 96% | No clear effect |
| Fine-tune Llama-3.1 8B Instruct on ExploreToM-generated data | Llama-3.1 8B Instruct · MMLU | accuracy: 69% -> 67% | Hurt |
| Use A* search instead of over-generation and filtering | ExploreToM story generation · 81 randomly selected ExploreToM settings | model accuracy on generated stories: 2 accuracy points | Helped |
| Add asymmetry modifiers to story generation (ablation) | ExploreToM · ExploreToM story structures across action sets | accuracy | Mixed |
| Apply story infilling to story structures (ablation) | ExploreToM · ExploreToM infilled stories | accuracy: average accuracy increased by 0.12 | Hurt |
| Increase the proportion of stories that require theory of mind in fine-tuning data (ablation) | ExploreToM-8B training · ToMi | accuracy | Helped |
Surprising findings
- Randomly sampled stories rarely require theory of mind (often 78%+ of stories are uninteresting), so training data must be explicitly biased toward ToM.
- Models perform notably worse even on pure state-tracking (uninteresting) questions, suggesting poor state tracking underlies ToM failures.
Limitations
- Requires manual coding of each new action and its belief-update semantics, limiting scalability.
- Stories are not goal-oriented and simplify real-world narratives.
- Fine-tuning used a maximum length of 300 tokens, which limits gains on long-context benchmarks like FANToM.
Experimental setup
Datasets: ExploreToM-generated stories ToMi Hi-ToM BigToM OpenToM FANToM Multi3Woz MMLU
Benchmarks: ExploreToM-generated data ToMi Hi-ToM BigToM OpenToM FANToM Multi3Woz MMLU
Metrics: accuracy F1
Baselines: ToMi Hi-ToM BigToM OpenToM FANToM Over-generation and filtering MMLU Multi3Woz
Builds on: ToMi Hi-ToM A* search AutoBencher
Closest papers by meaning
- OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling (NeurIPS 2026 · similarity 0.89)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.88)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.88)
- From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models (ICML 2025 · similarity 0.88)
- Position: Theory of Mind Benchmarks are Broken for Large Language Models (ICML 2025 · similarity 0.87)
- ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.