Research map / Vision-language and multimodal / Multimodal LLMs

On the Generalization Capacities of MLLMs for Spatial Intelligence

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Camera-Aware MLLM, which injects camera-ray embeddings, camera-aware geometric augmentation, and UniDepth v2 geometric prior distillation into a VG-LLM-style spatial MLLM to improve cross-camera generalization.

Problem

RGB-only MLLMs for spatial tasks such as 3D localization overfit to the training camera distribution and fail under intrinsics changes (e.g., resizing), since camera parameters are omitted.

Approach

Make spatial MLLMs camera-aware by conditioning every visual token on dense camera ray embeddings derived from intrinsics, augmenting training by synthetically perturbing intrinsics with consistent image transforms, and distilling geometric priors from a frozen monocular metric depth model.

Key results

• On the ScanNet-val x1.2 ablation, the full camera-aware model reaches 52.1 F1@0.25 versus 39.1 for the VG-LLM baseline (20 common classes).
• Ours-4B achieves 68.35 average on SPAR-Bench (full), above SPAR-8B at 63.25 and the listed baselines.
• Ours-4B reports 77.0 on BLINK-Spatial and 91.5 on CV-Bench-3D distance, with CV-Bench-3D average 90.7 slightly below VG-LLM-4B at 91.3.

Surprising findings

• Training on the mixed multi-source 3D dataset lowered ScanNet-val performance for Qwen2.5-VL-3B (47.5 to 38.4 F1), showing that scaling data across cameras can hurt.
• Simply resizing ScanNet test images by 0.8x or 1.2x drops F1 for both Qwen2.5-VL-3B and VG-LLM-4B substantially.

Limitations

• Ours-4B underperforms VG-LLM-4B on some benchmarks (e.g., CV-Bench-3D average, VSI-Bench route planning and appearance order), and the VSI-Bench appearance-order score drops to 20.2.
• Relies on a frozen UniDepth v2 teacher for geometric priors and on VG-LLM/VGGT components, adding dependencies.
• Theoretical analysis is limited to an ideal pinhole model without distortion and to fronto-parallel configurations.

Experimental setup

Datasets: ScanNet ARKitScenes Matterport3D 3RScan SUN RGB-D Objectron ScanRefer Scan2Cap LLaVA-Video-178k SPAR

Benchmarks: SPAR-Bench VSI-Bench CV-Bench-3D BLINK-Spatial ScanNet-val (cross-camera resize) TUM-RGBD (qualitative)

Metrics: F1@0.25 accuracy precision recall

Baselines: Qwen2.5-VL VG-LLM GPT-4o Gemini-2.5-Pro SPAR-8B SAT-LLaVA-Video-7B RynnEC-7B

Builds on: VG-LLM Metric3D UniDepth Qwen2.5-VL VGGT

Closest papers by meaning

• Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.91)
• SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.91)
• G$^2$VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning (CVPR 2026 · similarity 0.90)
• Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.90)
• Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models (NeurIPS 2025 · similarity 0.90)
• Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.89)