Research map / Vision-language and multimodal / Multimodal LLMs

3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes 3ViewSense, a two-stage SFT plus GRPO framework that trains Qwen3-VL-4B to first generate front/left/top orthographic view descriptions and then reason over them, and introduces the OrthoMind-3D benchmark for block counting and object reasoning.

Problem

VLMs fail on elementary spatial tasks such as counting occluded stacked blocks, lacking a view-consistent 3D mental representation built from 2D observations.

Approach

A "Simulate-and-Reason" pipeline: the model first induces canonical orthographic (front/left/top) views from an egocentric image via SFT (OMS), then performs view-grounded reasoning (VGR) conditioned on those views, refined with GRPO using strict or slack math-verifiable rewards.

Key results

• 3ViewSense-4B-rl-strict reaches 95.0% on in-domain block counting vs 6.2% for Qwen3-VL-4B-Instruct base in the Table 1 comparison.
• On external MindCube-Tiny, 3ViewSense-4B-rl-slack improves from 27.2 to 38.9 accuracy.
• Two-stage SFT (OMS then VGR) reaches 78.6 ID and 49.5 OOD on OrthoMind-3D, above VGR-SFT alone (70.3 / 46.6).

Surprising findings

• Direct QA SFT scores higher in-domain (80.3 vs 70.3) than 3ViewSense reasoning but collapses on external benchmarks (SPBench-SI 1.3 vs 21.2).
• Adding explicit three-view descriptions degrades some open models, e.g., GLM4.1V-9B on block counting (15.0 to 10.8).

Limitations

• Three fixed orthographic views cannot capture tasks needing physical or semantic priors such as support, affordances, or dynamics.
• Performance can degrade when view induction is unreliable.
• In-domain results are near-saturated, so the benchmark is less discriminative for the RL variants.

Experimental setup

Datasets: OrthoMind-3D MindCube-Tiny CV-Bench SPBench-SI OmniSpatial ViewSpatial

Benchmarks: OrthoMind-3D (ID) OrthoMind-3D (OOD) MindCube-Tiny CV-Bench 2D OmniSpatial SPBench-SI ViewSpatial

Metrics: accuracy (pass@1) average response length

Baselines: GPT-4o Gemini-2.0-Flash GPT-5 Gemini-3-pro Claude-Sonnet-4.5 SpatialLadder-3B Spatial-MLLM-4B SpaceOm-4B SpaceQwen2.5-VL-3B-Instruct Qwen3-VL-8B-Instruct Qwen3-VL-4B-Instruct InternVL3.5-8B

Builds on: GRPO Qwen3-VL LLaMA-Factory verl

Closest papers by meaning

• Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views (CVPR 2026 · similarity 0.92)
• Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.92)
• Pursuing Minimal Sufficiency in Spatial Reasoning (ICLR 2026 · similarity 0.92)
• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.92)
• SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.91)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.91)