AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Perception, Not Reasoning, Limits Visual Theory of Mind

Mohamed Rayan Barhdadi, Syed Talal Wasim, Jürgen Gall, Erchin Serpedin, HASAN KURBAN

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Argues that the visual false-belief performance gap of vision-language models stems from perception rather than belief reasoning, and proposes a five-condition dissociation protocol to test this.

Problem

Vision-language models pass text-based false-belief probes near human levels but drop by 20+ percentage points when the same problem is presented visually; the consensus attributes this to a conceptual reasoning deficit.

Approach

Proposes that the bottleneck is perceptual (what the model cannot see) rather than conceptual (what it cannot reason about), and tests this with a five-condition dissociation protocol that holds the reasoning problem fixed while varying the visual presentation.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add full ground-truth scaffold (B_f) to imageVLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 73.9 -> 98.4Improved
add full ground-truth scaffold (B_f) to imageVLMs · MuMA-ToM (900 questions)accuracy (%): 44.7 -> 58.1Improved
add full ground-truth scaffold (B_f) to imageVLMs · MindPower (overall layer-mean)layer-mean score (%): 19.1 -> 27.3Improved
add automatic-parser scaffold (B_a) to imageVLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 73.9 -> 75.7No clear effect
add automatic-parser scaffold (B_a) to imageVLMs · MindPower (overall layer-mean)layer-mean score (%): 19.1 -> 18.3No clear effect
add perceptual-grade scaffold (B_p) to image (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 73.9 -> 88.8Improved
add visibility and absence statements to B_p (B_p to B_f) (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 88.8 -> 98.4Improved
remove visibility statements from B_f (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 98.4 -> 90.0Hurt
add cross-frame event structure to static positions (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 76.7 -> 88.8Improved
add per-panel positions to entity lists (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 74.0 -> 76.7No clear effect
replace B_p with length-matched corrupted scaffolds (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): 88.8 -> 74.7 / 79.5 / 80.6Hurt
add chain-of-thought prompting (visual only) (ablation)VLMs · Abstract 2D Stage 3 (False Belief)accuracy (%): -1.7 ppNo clear effect

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy