Research map / Vision-language and multimodal / Multimodal LLMs

GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GeoSense, a Qwen2.5-VL-3B-based MLLM with an independent VGGT geometry input channel that learns, via alignment and spatial-aware SFT on model-adaptive data, when to request geometric tokens.

Problem

Existing MLLMs fuse 3D geometry features into every input regardless of necessity, which adds overhead and can degrade general 2D reasoning; spatial reasoning needs geometry only in some cases.

Approach

Treat geometry as an independent, on-demand input modality (boundary-delimited tokens) and train the model to emit a <vggt> trigger only when its own with/without-geometry performance discrepancy shows geometry is needed, using model-derived labels instead of hand-crafted rules.

Key results

• GeoSense (3B) reaches Spatial Avg. 56.6 vs 43.4 for Qwen2.5-VL-3B and overall average 55.9, ranked first in the main table.
• In the ablation, the adaptive trigger activates VGGT tokens for 35.68% of samples, versus 100% for always-on fusion (VG-LLM), and gets VSI-Bench avg 54.86 vs 52.34.
• Perception SFT raises the spatial average from 51.0 to 63.4 across the four-stage ablation while MME recovers from 1803 to 1922 after alignment-only training.

Surprising findings

• Injecting geometry features hurt some general samples: about 3% of mixed samples degraded when 3D features were added, and scaling rigid fusion (VG-LLM) to 940k samples dropped POPE from 86.9 to 74.2.
• Alignment SFT alone degraded general benchmarks (MME fell to 1803) before perception tuning restored them.

Limitations

• Relies primarily on the VGGT encoder; other 3D representations such as point clouds are not explored.
• Rotation task on MindCube degrades with VGGT features, attributed to wide-angle sparse views hindering VGGT global representations.
• Embodied benchmark results show dynamic scenarios lag, attributed to fine-tuning data bias toward static scenes.

Experimental setup

Datasets: VSI-590K SophiaVL-R1-130k Mantis-Instruct LLaVA-Hound-64k Spar-234k VSI-Bench MMSI-Bench MindCube-Tiny BLINK CV-Bench EmbSpatial MMBench

Benchmarks: VSI-Bench MMSI-Bench MindCube-Tiny BLINK CV-Bench (3D) EmbSpatial MMBench MME POPE WeMath Quantiphy

Metrics: accuracy average score

Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B InternVL3-2B ViLaSR-7B SpaceR-sft-7B SpatialLadder-3B Cambrian-S-3B VG-LLM VST-3B-SFT

Builds on: VG-LLM Qwen2.5-VL VGGT Spatial-MLLM

Closest papers by meaning

• Thinking with Geometry: Active Geometry Integration for Spatial Reasoning (ICML 2026 · similarity 0.92)
• From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs (CVPR 2026 · similarity 0.91)
• GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning (CVPR 2026 · similarity 0.90)
• 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models (ICML 2026 · similarity 0.90)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing (NeurIPS 2025 · similarity 0.90)
• SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.90)