Research map / Vision-language and multimodal / Multimodal LLMs

SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SpaceMind, an RGB-only VLM that fuses VGGT geometry tokens with InternViT visual tokens via a Camera-Guided Modality Fusion module, reaching state-of-the-art on VSI-Bench, SQA3D and SPBench.

Problem

Vision-language models struggle with 3D spatial reasoning (distance, size, cross-view consistency) from RGB input, and existing geometry-augmented fusion methods conflate camera and scene signals.

Approach

Treat camera tokens as a dedicated guiding modality: camera-conditioned bias on spatial keys/values, a query-independent per-token importance weight from the spatial branch, and a camera-conditioned SwiGLU gate on the fused visual tokens, all inserted before the LLM.

Key results

• SpaceMind reaches 69.6 average on VSI-Bench, +8.7 over the prior best VLM-3R (60.9).
• On SQA3D test, SpaceMind reports EM@1 61.1 and EM@R1 63.8 using video-only input, a new state of the art.
• On SPBench overall, SpaceMind scores 67.3 versus 54.0 for the best prior spatial model (VILASR-7B).

Surprising findings

• Adding VGGT tokens with shallow cross-attention raises the InternVL3-8B fine-tuned average from 63.07 to 66.77, and the twMLP module adds only +0.40 over that shallow baseline.

Limitations

• Relies on frozen VGGT and visual encoders, so geometry quality is bounded by the pretrained spatial encoder.
• Route planning remains roughly on par with prior SOTA (44.3 vs 45.4 for VLM-3R), with no clear gain.
• SPBench-SI MCQ accuracy (53.2) trails ViLaSR and Qwen2.5-VL-7B.

Experimental setup

Datasets: VLM-3R-data ViCA-322K SQA3D

Benchmarks: VSI-Bench SQA3D SPBench

Metrics: relative accuracy mean accuracy EM@1 EM@R1

Baselines: GPT-4o Gemini-1.5 Pro InternVL3-78B LLaVA-OneVision-72B SpaceR ViLaSR Spatial-MLLM VLM-3R Video-3D LLM Gemini-2.0-Flash Qwen2.5-VL-7B Kimi-VL-A3B

Builds on: VGGT InternVL3 SwiGLU VLM-3R

Closest papers by meaning

• SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.92)
• SpatialStack: Layered Geometry-Language Fusion for 3D VLM Spatial Reasoning (CVPR 2026 · similarity 0.91)
• From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
• SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning (NeurIPS 2025 · similarity 0.91)
• On the Generalization Capacities of MLLMs for Spatial Intelligence (ICLR 2026 · similarity 0.91)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.91)