Research map / Vision-language and multimodal / Multimodal LLMs
Perception, Not Reasoning, Limits Visual Theory of Mind
Mohamed Rayan Barhdadi, Syed Talal Wasim, Jürgen Gall, Erchin Serpedin, HASAN KURBAN
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that the visual false-belief performance gap of vision-language models stems from perception rather than belief reasoning, and proposes a five-condition dissociation protocol to test this.
Problem
Vision-language models pass text-based false-belief probes near human levels but drop by 20+ percentage points when the same problem is presented visually; the consensus attributes this to a conceptual reasoning deficit.
Approach
Proposes that the bottleneck is perceptual (what the model cannot see) rather than conceptual (what it cannot reason about), and tests this with a five-condition dissociation protocol that holds the reasoning problem fixed while varying the visual presentation.
Key results
- Vision-language models pass text-based false-belief probes near human levels but drop by twenty or more percentage points when the same problem is conveyed visually (as stated in the abstract's motivation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add full ground-truth scaffold (B_f) to image | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 73.9 -> 98.4 | Improved |
| add full ground-truth scaffold (B_f) to image | VLMs · MuMA-ToM (900 questions) | accuracy (%): 44.7 -> 58.1 | Improved |
| add full ground-truth scaffold (B_f) to image | VLMs · MindPower (overall layer-mean) | layer-mean score (%): 19.1 -> 27.3 | Improved |
| add automatic-parser scaffold (B_a) to image | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 73.9 -> 75.7 | No clear effect |
| add automatic-parser scaffold (B_a) to image | VLMs · MindPower (overall layer-mean) | layer-mean score (%): 19.1 -> 18.3 | No clear effect |
| add perceptual-grade scaffold (B_p) to image (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 73.9 -> 88.8 | Improved |
| add visibility and absence statements to B_p (B_p to B_f) (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 88.8 -> 98.4 | Improved |
| remove visibility statements from B_f (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 98.4 -> 90.0 | Hurt |
| add cross-frame event structure to static positions (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 76.7 -> 88.8 | Improved |
| add per-panel positions to entity lists (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 74.0 -> 76.7 | No clear effect |
| replace B_p with length-matched corrupted scaffolds (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): 88.8 -> 74.7 / 79.5 / 80.6 | Hurt |
| add chain-of-thought prompting (visual only) (ablation) | VLMs · Abstract 2D Stage 3 (False Belief) | accuracy (%): -1.7 pp | No clear effect |
Surprising findings
- The visual theory-of-mind deficit is attributed to perception rather than the belief-reasoning deficit that the consensus assumes.
Closest papers by meaning
- Imagination Helps Visual Reasoning, But Not Yet in Latent Space (ICML 2026 · similarity 0.89)
- On the Faithfulness of Visual Thinking: Measurement and Enhancement (NeurIPS 2026 · similarity 0.89)
- MentisOculi: Revealing the Limits of Reasoning with Mental Imagery (ICML 2026 · similarity 0.88)
- Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators (NeurIPS 2026 · similarity 0.88)
- The Perceptual Bandwidth Bottleneck in Vision-Language Models: Active Visual Reasoning via Sequential Experimental Design (ICML 2026 · similarity 0.88)
- Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
