AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Understanding VLMs Spatial Mental Modeling Capability from Limited Views

Qineng Wang, Baiqiao Yin, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu and 2 more

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MindCube, a benchmark of 21,154 questions over 3,268 images testing whether VLMs can form spatial mental models from limited views, and finds existing VLMs near random.

Problem

Vision language models may fail to build internal spatial mental models of unseen space from only a few views, which humans do for reasoning about layout, perspective, and motion.

Approach

A new benchmark (MindCube) with 21,154 questions over 3,268 multi-view images to systematically probe spatial mental modeling from limited views; existing VLMs perform near random on it.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
insert interpolated views between the given views (ablation)frozen Qwen2.5-VL-3B · MindCube-Tinyaccuracy: 37.81 -> 37.90No clear effect
supply a ground-truth augmented cognitive map as input without reasoning (ablation)frozen Qwen2.5-VL-3B · MindCube-Tinyaccuracy: 37.81 -> 32.00Hurt
add free-form reasoning at inference (ablation)frozen Qwen2.5-VL-3B · MindCube-Tinyaccuracy: 37.81 -> 40.48Improved
generate a cognitive map before reasoning (ablation)frozen Qwen2.5-VL-3B · MindCube-Tinyaccuracy: 40.48 -> 41.33 (plain map); Rotation 32.00 -> 25.00Mixed
SFT on raw QA pairsQwen2.5-VL-3B · MindCube-Tinyaccuracy: 37.81 -> 52.67Improved
SFT with augmented-map output only, instead of raw QA (ablation)Qwen2.5-VL-3B SFT · MindCube-Tinyaccuracy: 52.67 -> 52.48No clear effect
SFT with free-form reasoning added (ablation)Qwen2.5-VL-3B SFT · MindCube-Tinyaccuracy: 52.67 -> 55.43Improved
SFT to generate a plain cognitive map then reason (ablation)Qwen2.5-VL-3B SFT · MindCube-Tinyaccuracy: 52.67 -> 57.81Improved
train RL from scratch with sparse reward instead of from SFT checkpoint (ablation)RL-FFR · MindCube-Tinyaccuracy: 59.14 -> 49.52Hurt
initialize RL from the best SFT checkpoint (ablation)RL-Plain-CGMap-FFR-Out · MindCube-Tinyaccuracy: 50.86 -> 61.33Improved
fix object order in cognitive map supervision instead of randomizing (ablation)Plain-CGMap-FFR-Out SFT · MindCube-Tinyaccuracy: 57.81 -> 60.76Improved
use map-then-reason rather than reasoning only in RL from scratch (ablation)RL from scratch · MindCube-Tinyaccuracy: 49.52 -> 52.48 (Aug), 50.86 (Plain)Mixed

Surprising findings

Experimental setup

Benchmarks: MindCube

Metrics: accuracy

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy