Research map / Large language models / Agents, code and math
Language Models Use Lookbacks to Track Beliefs
Nikhil Prakash, Natalie Shapira, Arnab Sen Sharma, Christoph Riedl, Yonatan Belinkov, Tamar Rott Shaham, David Bau, Atticus Geiger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| instruction tuning of Qwen2.5-7B | Qwen2.5-7B · CausalToM no-visibility | accuracy: 0.280 -> 0.948 | Improved |
| instruction tuning of Qwen2.5-14B | Qwen2.5-14B · CausalToM visibility | accuracy: 0.433 -> 0.912 | Improved |
| instruction tuning of Llama-3.1-8B | Llama-3.1-8B · CausalToM visibility | accuracy: 0.293 -> 0.310 | Mixed |
| scale from 8B to 70B Llama-3 Instruct | Meta-Llama-3-Instruct · CausalToM visibility | accuracy: 0.085 -> 0.923 | Improved |
| add visibility sentence between characters | LMs · CausalToM | accuracy: 0.948 -> 0.719 | Hurt |
| intervene on both recalled and lookback tokens rather than only one (ablation) | Visibility lookback patching · Visibility interchange intervention | alignment | Improved |
| freeze the state token residual stream while swapping character and object tokens (ablation) | Source reference patching · Source reference localization | alignment | Improved |
| patch the full residual stream instead of the identified subspace (ablation) | Answer lookback patching · CausalToM belief tracking (interchange interventions, n=80) | intervention success | No clear effect |
Closest papers by meaning
- OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling (NeurIPS 2026 · similarity 0.89)
- How do Language Models Bind Entities in Context? (ICLR 2024 · similarity 0.87)
- Monitoring Latent World States in Language Models with Propositional Probes (ICLR 2025 · similarity 0.87)
- Reflection-Bench: Evaluating Epistemic Agency in Large Language Models (ICML 2025 · similarity 0.86)
- ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind (AAAI 2025 · similarity 0.86)
- Language Models Represent Beliefs of Self and Others (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
