Research map / Vision-language and multimodal / Multimodal LLMs

Thinking with Geometry: Active Geometry Integration for Spatial Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GeoThinker, which injects VGGT geometry features into Qwen-VL LLM layers via frame-wise cross-attention with importance gating and a global scalar, reaching 72.6 on VSI-Bench.

Problem

Existing geometry-aware MLLMs fuse 3D encoder features passively and indiscriminately, causing semantic-geometry misalignment and redundant signals in spatial reasoning.

Approach

Spatial-Grounded Fusion lets semantic image hidden states in selected mid-depth LLM layers query geometry tokens from the same frame via cross-attention, with a learned importance score added as an attention bias and a tanh-gated learnable scalar controlling injection strength.

Key results

• GeoThinker (Qwen3VL-8B, 32-frame) reaches 72.6 on VSI-Bench, and 68.1 on VSI-Debiased at 128 frames, above Cambrian-S-7B (59.9).
• Vanilla regime: GeoThinker Qwen2.5VL-3B and 7B reach 48.9 and 50.5 average on VSI-Bench, versus 46.7 (VG-LLM-4B) and 49.7 (VG-LLM-8B).
• On NAVSIM navtest, PDMS rises from 81.6 (ReCogDrive) to 83.6; on RefSpatial-Bench average accuracy rises from 42.56 to 44.22 over RoboRefer.

Surprising findings

• Applying spatial-grounded fusion to all LLM layers (rho=1.0) collapses performance to near zero (0.41 average) on VSI-Bench, attributed to interference with EOS token prediction.
• Cambrian-S-7B loses 4.1 points on VSI-Bench when general video data is added, while GeoThinker gains 0.6 points.

Limitations

• Performance collapses when fusion is applied to all layers, so layer selection is sensitive and must be tuned.
• Trails proprietary models such as Gemini 3 Pro on MMSI, BLINK, and 3DSRBench per the authors' EASI analysis.
• Inference is slower than the base Qwen-VL because VGGT runs sequentially before the LLM.

Experimental setup

Datasets: VSI-590K SPAR-7M VLM-3R MindCube LLaVA-Video-178K (LLaVA-Hound) PhysGame Cambrian-S 3M general video subset

Benchmarks: VSI-Bench VSI-Debiased MMSI-Bench MindCube ViewSpatial SITE CV-Bench VideoMME MVBench RefSpatial-Bench NAVSIM navtest VSTI-Bench

Metrics: accuracy average score PDMS NC DAC TTC Comf EP

Baselines: VG-LLM Cambrian-S VLM-3R 3DThinker SenseNova-SI Spatial-MLLM SpatialLadder RoboRefer ReCogDrive

Builds on: VG-LLM VGGT Qwen2.5-VL Qwen3-VL RoboRefer ReCogDrive

Closest papers by meaning

• Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views (CVPR 2026 · similarity 0.93)
• GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning (ICML 2026 · similarity 0.92)
• EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.90)
• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.90)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.90)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.90)