Research map / Large language models / Language, safety and interpretability
ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds ToMATO, a multiple-choice theory-of-mind benchmark of 5.4k questions over 753 LLM-LLM role-play conversations where agents verbalize first- and second-order mental states as hidden "inner speech" thoughts, with big-five personality conditions and a false-belief subset.
Problem
Existing ToM benchmarks cover few mental state types, rarely test false beliefs beyond beliefs, and ignore character personality traits, so they diverge from real-world social conversation.
Approach
Role-playing LLM pairs converse with information asymmetry: each agent verbalizes its thoughts in parentheses before each utterance, hidden from the other agent, and these thoughts serve as gold answers for questions on five mental state types; personality traits (big five patterns) and goals are hidden per agent to induce false beliefs.
Key results
- GPT-4o mini scores 73.5% overall on ToMATO versus 87.3% for human annotators; Llama-3.1-70B reaches 77.9% overall, the best open model reported.
- On the false-belief subset, all models drop further; GPT-4o mini reaches 60.2% on second-order FB beliefs versus 84.4% for humans, and some small models fall below the lexical overlap baseline.
- Fine-tuning Llama-3-8B with QLoRA on 7k generated examples raises ToMATO ID accuracy from 50.3% to 74.6% but lowers SocialIQa accuracy from 63.9% to 46.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model scale from Llama-3-8B to Llama-3-70B | Llama-3 on ToMATO · ToMATO ALL | accuracy (%): 47.5{ $\pm$0.2} -> 76.0{ $\pm$0.2} | Helped |
| Use zero-shot chain-of-thought prompting (ablation) | Llama-3-8B · ToMATO ALL | accuracy (%): 47.5{ $\pm$0.2} -> 60.4{ $\pm$0.5} | Helped |
| Use zero-shot chain-of-thought prompting (ablation) | Llama-3-70B · ToMATO ALL | accuracy (%): 76.0{ $\pm$0.2} -> 76.5{ $\pm$0.3} | No clear effect |
| Use zero-shot chain-of-thought prompting (ablation) | Mistral-7B · ToMATO ALL | accuracy (%): 60.9{ $\pm$0.1} -> 61.4{ $\pm$0.2} | No clear effect |
| Use zero-shot chain-of-thought prompting (ablation) | GPT-3.5-Turbo · ToMATO ALL | accuracy (%): 55.8{ $\pm$0.3} -> 61.3{ $\pm$2.6} | Helped |
| Fine-tune on generated ToM conversations (ablation) | Llama-3-8B · ToMATO ID | accuracy: 50.3 -> 74.6 | Helped |
| Fine-tune on generated ToM conversations (ablation) | Llama-3-8B · ToMATO OOD | accuracy: 44.8 -> 67.9 | Helped |
| Fine-tune on generated ToM conversations (ablation) | Llama-3-8B · SocialIQa | accuracy: 63.9 -> 46.7 | Hurt |
| Hide both the system prompt and thoughts from the other agent (ablation) | ToMATO conversation generation · False belief frequency (GPT judge) | frequency probability (%): 39.0 -> 46.6 | Helped |
| Hide both the system prompt and thoughts from the other agent (ablation) | ToMATO conversation generation · False belief frequency (Human judge) | frequency probability (%): 30.5 -> 51.0 | Helped |
| Add system-prompt information asymmetry while retaining thought information asymmetry (ablation) | ToMATO conversation generation · False belief frequency (GPT judge) | frequency probability (%): 40.4 -> 46.6 | Helped |
| Add system-prompt information asymmetry while retaining thought information asymmetry (ablation) | ToMATO conversation generation · False belief frequency (Human judge) | frequency probability (%): 32.0 -> 51.0 | Helped |
Surprising findings
- Understanding desires is easier than beliefs for LLMs, consistent with children.
- Fine-tuning on ToMATO-style data improves in-domain ToM accuracy but degrades general social reasoning on SocialIQa, suggesting overfitting.
Limitations
- Llama-3-70B-Instruct generated the benchmark, so comparisons with Llama models are unfair.
- Conscientiousness is less reliably reflected in generated outputs (60-68% success in pairwise comparison).
- Benchmark is multiple-choice and relies on generated thoughts as gold answers, with 5.8% of conversations excluded for quality issues.
Experimental setup
Datasets: ToMATO SocialIQa SOTOPIA scenarios
Benchmarks: ToMATO ToMATO-FB SocialIQa
Metrics: accuracy
Baselines: Llama-3-8B-Instruct Llama-3-70B-Instruct Llama-3.1-8B-Instruct Llama-3.1-70B-Instruct Gemma-2-9B-it Mistral-7B-Instruct Mixtral-8x7B-Instruct GPT-3.5-Turbo GPT-4o mini lexical overlap
Builds on: SOTOPIA FANToM Chain-of-thought prompting QLoRA
Closest papers by meaning
- OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling (NeurIPS 2026 · similarity 0.91)
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.89)
- Position: Theory of Mind Benchmarks are Broken for Large Language Models (ICML 2025 · similarity 0.88)
- Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · similarity 0.88)
- Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models (AAAI 2025 · similarity 0.87)
- Explore Theory of Mind: program-guided adversarial data generation for theory of mind reasoning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.