Research map / Vision-language and multimodal / Multimodal LLMs

Multi-view Relational Distillation for Spatial Reasoning with Vision-Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes multi-view relational distillation (MVRD), which distills patch-wise cross-view cosine-similarity structure from the geometry teacher VGGT into a VLM instead of matching features, improving VSI-Bench spatial reasoning while preserving vision-language alignment.

Problem

Feature distillation from geometry-grounded vision models improves spatial reasoning in VLMs but disrupts pretrained vision-language alignment, degrading linguistic grounding tasks such as object counting, size and appearance order.

Approach

Instead of matching multi-view teacher features, distill the patch-wise cosine similarity matrix across views (with causal masking, zeroed diagonal, and a row-wise Pearson correlation loss); similarity matching leaves features underdetermined so the student can stay near its pretrained space, and a dual-pathway LoRA split separates geometric and semantic adapters.

Key results

• MVRD reaches 59.9 average on VSI-Bench with LLaVA-Video-7B versus 57.7 for SFT and 57.0 for 3DRS*, and MVRD + Dual Pathway reaches 60.4, within 0.5 points of VLM-3R (60.9).
• MVRD + Dual Pathway adds 0.19B parameters (about 25% of CUT3R) and incurs about 16% of CUT3R's added latency on 32 views.
• On 3D scene understanding, MVRD achieves 63.3 Acc@0.25 on ScanRefer and 61.1 F1@0.25 on Multi3DRefer, improving over 3DRS on all reported metrics.

Surprising findings

• Feature distillation degrades linguistic-grounding tasks such as appearance order (19.6 vs 38.5 for SFT on VLM-3R data), despite improving geometric tasks.
• Early and middle-layer distillation underperforms the SFT-only baseline, and only later-layer distillation helps.

Limitations

• Training is at moderate scale, which still induces some trade-off between geometric and linguistic capabilities.
• Geometric-centered gains are smaller with the dual pathway, trading some geometric accuracy for linguistic balance.
• Distillation relies on a frozen geometric teacher (VGGT) and fixed-frame-index augmentation was needed to avoid memorizing per-index relations.

Experimental setup

Datasets: VLM-3R spatial reasoning dataset ScanNet ScanNet++ ARKitScenes ScanRefer Multi3DRefer Scan2Cap ScanQA SQA3D

Benchmarks: VSI-Bench ScanRefer Multi3DRefer Scan2Cap ScanQA SQA3D

Metrics: accuracy (MCA) MRA Acc@0.25 Acc@0.5 F1 CIDEr BLEU-4 EM text-grounded object detection precision

Baselines: SFT 3DRS feature distillation VLM-3R VG-LLM-8B GPT-4o Gemini-2.5 Pro InternVL3-78B Ross3D Video-3D LLM 3D-LLaVA Inst3D-LLM

Builds on: 3DRS VGGT VLM-3R LoRA relational knowledge distillation

Closest papers by meaning

• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.91)
• Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning (CVPR 2026 · similarity 0.91)
• Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models (CVPR 2026 · similarity 0.91)
• G$^2$VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning (CVPR 2026 · similarity 0.91)
• From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
• SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning (NeurIPS 2025 · similarity 0.91)