Research map / Vision-language and multimodal / Multimodal LLMs

Reasoning in Space via Grounding in the World

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GS-Reasoner, a 3D LLM with a semantic-geometric hybrid patch representation (dual-path pooling) that performs autoregressive 3D grounding as chain-of-thought for spatial reasoning, trained with a new GCoT dataset.

Problem

3D LLMs lack a unified representation that jointly captures semantic and geometric information, so they either ground poorly or depend on external detectors/decoders, which hinders integrating grounding into spatial reasoning.

Approach

A dual-path pooling module fuses point-cloud geometric features with patch semantic features (cross-attention path) and 3D position (interpolation at the patch-center point) into one image-patch-based hybrid representation, enabling autoregressive bbox output without extra tokens or external modules; a GCoT dataset interleaves 3D grounding into reasoning chains generated by GPT-4o.

Key results

• GS-Reasoner achieves 60.8 Acc@25 and 42.2 Acc@50 on ScanRefer and 56.7 Acc@25 on SR3D, matching 3D LLMs with external grounding modules without external components.
• On VSI-Bench, GS-Reasoner (predicted depth) reaches 64.7 average, and 70.1 with ground-truth depth, about 4 points above the prior best VLM-3R-7B (60.9) under predicted depth.
• On Scan2Cap, GS-Reasoner reaches CIDEr 101.0 and B-4 47.6, outperforming ROSS3D (81.3 CIDEr, 43.4 B-4).

Surprising findings

• Naive max/mean pooling of geometric features per patch yields poor grounding, and average positional encoding as in Video-3D LLM overfits badly (15.4 Acc@25 without augmentation).
• Without data augmentation the autoregressive model overfits object grounding, and LLaVA-NeXT without 3D inputs gets 0.0 Acc@25 on ScanRefer.

Limitations

• Trails mesh-proposal-based 3D LLMs and expert models on ScanRefer Acc@50, SR3D and NR3D, attributed to noisy sensor point clouds and bbox-only supervision.
• Does not achieve leading results on ScanQA and SQA3D, attributed to ambiguous questions and answer-distribution bias.
• Route planning accuracy is weaker than some baselines (44.3 vs 45.4 VLM-3R-7B in the reported table).

Experimental setup

Datasets: ScanRefer Multi3DRef SR3D NR3D ScanQA SQA3D Scan2Cap ScanNet ScanNet++ ARKitScenes VSI-Bench-style QA generation

Benchmarks: ScanRefer Multi3DRef SR3D NR3D VSI-Bench Scan2Cap ScanQA SQA3D Locate 3D dataset (zero-shot)

Metrics: Acc@25 Acc@50 F1@25 F1@50 accuracy CIDEr BLEU-4 METEOR ROUGE EM

Baselines: 3D-VisTA PQ3D UniVLG Locate 3D Chat-Scene Inst3D-LMM Video-3D LLM ROSS3D SeeGround Grounded 3D-LLM ReGround3D LLaVA-3D

Builds on: LLaVA-Video Point Transformer v3 Sonata VSI-Bench Video-3D LLM

Closest papers by meaning

• S$^2$-MLLM: Boosting Spatial Reasoning Capability of MLLMs for 3D Visual Grounding with Structural Guidance (CVPR 2026 · similarity 0.91)
• ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning (CVPR 2025 · similarity 0.91)
• SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning (NeurIPS 2025 · similarity 0.90)
• No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers (ICLR 2026 · similarity 0.90)
• From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.90)
• Grounded 3D-Aware Spatial Vision-Language Modeling (CVPR 2026 · similarity 0.90)