Research map / Large language models / Language, safety and interpretability
Language Models Represent Beliefs of Self and Others
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Probes linear belief states of protagonist and oracle from LLM attention heads, then steers those heads to change ToM accuracy on BigToM with Mistral-7B and DeepSeek-7B.
Problem
Whether LLMs internally represent their own and other agents' beliefs, and whether these representations causally affect theory-of-mind (ToM) reasoning performance.
Approach
Linear probes on per-head attention activations decode both the oracle's and protagonist's belief status; intervening along the joint-belief probe direction (boosting protagonist belief while separating it from oracle belief) improves false-belief ToM accuracy.
Key results
- Linear probes on single attention heads in middle layers decode the protagonist's belief status above 80% validation accuracy (Mistral-7B, Forward Belief), and oracle belief at about 97%.
- Intervening along the (+T_pF_o) joint-belief direction raises Mistral-7B Forward Belief false-belief accuracy from 0.33 to 0.66 and Both from 0.32 to 0.58.
- Transferring the Forward Belief direction to ToMi raises Mistral-7B false-belief ToM accuracy from 63.8 to 71.0 while MMLU and GLUE-style tasks change little.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Intervene along the joint protagonist-true/oracle-false belief direction | Mistral-7B · BigToM Forward Belief (Both) | accuracy: 0.31 -> 0.58 | Helped |
| Intervene along the joint protagonist-true/oracle-false belief direction | Mistral-7B · BigToM Forward Action (Both) | accuracy: 0.26 -> 0.31 | Helped |
| Intervene along the joint protagonist-true/oracle-false belief direction | Mistral-7B · BigToM Backward Belief (Both) | accuracy: 0.16 -> 0.41 | Helped |
| Intervene along the joint protagonist-true/oracle-false belief direction | DeepSeek-7B · BigToM Forward Belief (Both) | accuracy: 0.37 -> 0.38 | No clear effect |
| Intervene along the joint protagonist-true/oracle-false belief direction | DeepSeek-7B · BigToM Forward Action (Both) | accuracy: 0.31 -> 0.39 | Helped |
| Intervene along the joint protagonist-true/oracle-false belief direction | DeepSeek-7B · BigToM Backward Belief (Both) | accuracy: 0.29 -> 0.31 | Helped |
| Transfer the BigToM joint belief direction intervention | Mistral-7B · ToMi (FB) | ToM accuracy: 63.8 -> 71.0 | Helped |
| Transfer the BigToM joint belief direction intervention | Mistral-7B · ToMi (All) | accuracy: 66.0 -> 67.9 | Helped |
| Use random activation directions instead of belief directions (ablation) | Mistral-7B · BigToM Forward Belief (Both) | accuracy: 0.31 -> 0.32 | No clear effect |
| Intervene along the protagonist belief direction instead of the joint belief direction (ablation) | Mistral-7B · BigToM Forward Belief (Both) | accuracy: 0.31 -> 0.29 | Hurt |
| Remove the oracle belief direction by intervening in the -oracle direction (ablation) | Mistral-7B · BigToM Forward Belief (Both) | accuracy: 0.31 -> 0.41 | Helped |
| Use random activation directions instead of belief directions (ablation) | DeepSeek-7B · BigToM Forward Belief (Both) | accuracy: 0.37 -> 0.40 | Helped |
Surprising findings
- Steering toward protagonist belief alone barely changes performance, while reversing the oracle belief direction produces a noticeable change, suggesting oracle belief signals dominate the representation.
- Directions found in Forward Belief transfer to Forward Action and Backward Belief tasks, improving false-belief accuracy in both.
Limitations
- Evaluated only two 7B open models and specific social reasoning tasks, so generalization to other LLMs is untested.
- Intervention increases invalid (unparseable) responses, and true-belief accuracy slightly decreases.
Experimental setup
Datasets: BigToM ToMi MMLU CoLA MRPC QNLI
Benchmarks: BigToM ToMi MMLU CoLA MRPC QNLI
Metrics: accuracy probe accuracy true belief accuracy false belief accuracy both accuracy
Baselines: Mistral-7B (baseline) DeepSeek-7B (baseline) LLaMA-65b GPT-4 Claude-2 GPT-3.5 text-davinci-003
Builds on: inference-time intervention BigToM
Closest papers by meaning
- MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems (NeurIPS 2025 · similarity 0.90)
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.90)
- OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling (NeurIPS 2026 · similarity 0.89)
- Position: Theory of Mind Benchmarks are Broken for Large Language Models (ICML 2025 · similarity 0.88)
- AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.88)
- From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.