Research map / Vision-language and multimodal / Multimodal LLMs

G$^2$VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes G^2VLM, a 2B VLM with a dual-expert (geometry + semantic) Mixture-of-Transformer design that predicts 3D geometry (depth, point maps, camera poses) and improves spatial reasoning via shared attention and interleaved reasoning.

Problem

Vision-language models lack robust spatial understanding and reasoning because they process multi-view images as flat 2D sequences without learning visual geometry for 3D reconstruction.

Approach

Adds a geometric perception expert (DINOv2 features, global attention, lightweight 3D heads trained from scratch on 3D-annotated data) alongside the Qwen2-VL semantic expert in a Mixture-of-Transformer-Experts with shared self-attention, so the VLM can both reconstruct 3D and use geometry features for spatial QA via in-context learning.

Key results

• G^2VLM-SR-2B reaches 54.87 average on SPAR-Bench, versus 36.39 for GPT-4o (about 18.5 points higher).
• On monocular depth (Sintel), G^2VLM achieves Abs Rel 0.297 versus VGGT's 0.335, while pi^3 reports 0.277.
• G^2VLM-SR-2B scores 48.33 on MindCube and 37.02 on OST-Bench subset, the best among the compared open-source and spatial expert models on MindCube, with OST-Bench led by Qwen2.5-VL-72B.

Surprising findings

• DINO features from the geometry encoder also improve spatial understanding, not only geometry, when paired with a CLIP-style semantic encoder.
• Frozen-expert CE-only joint training is chosen for the main model, although VG+CE joint training yields the strongest results.

Limitations

• Training instability when scaling to larger models, as acknowledged in the conclusion.
• Large-scale 3D annotated data is needed for joint VG+CE training, limiting scalability.
• Lags on OST-Bench compared with the much larger Qwen2.5-VL-72B, suggesting knowledge capacity limits at 2B.

Experimental setup

Datasets: ScanNet Co3Dv2 BlendMVS DL3DV MegaDepth ScanNetPP SPAR-7M OmniSpatial MindCube OST-Bench LLaVA-OneVision

Benchmarks: Sintel NYU-v2 ETH3D 7-Scenes Co3Dv2 SPAR-Bench MindCube OST-Bench OmniSpatial

Metrics: Abs Rel delta<1.25 Accuracy Completion RRA@30 RTA@30 AUC@30 accuracy

Baselines: VGGT pi^3 CUT3R Fast3R FLARE GPT-4o Qwen2.5-VL-72B Spatial-MLLM-7B VLM-3R-7B SpaceQwen2.5-VL-3B

Builds on: Qwen2-VL VGGT pi^3 Bagel DUSt3R

Closest papers by meaning

• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.92)
• Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.92)
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models (NeurIPS 2025 · similarity 0.92)
• From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.92)
• HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models (CVPR 2026 · similarity 0.92)
• Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.92)