Research map / Vision-language and multimodal / Multimodal LLMs

SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpatialScore, a 5K-sample, 30-task spatial intelligence benchmark evaluating 49 MLLMs, plus SpatialCorpus (331K QA SFT data) and training-free SpatialAgent with 12 spatial perception tools.

Problem

Existing MLLM spatial intelligence evaluations are fragmented, simplistic, and narrow in modality and question format, and MLLMs lag far behind humans on spatial understanding.

Approach

A holistic benchmark combining repurposed 3D-annotation QA with curated samples from 23 datasets, plus two enhancement routes: SFT on a large synthetic spatial corpus and a training-free multi-agent (Plan-Execute and ReAct) framework that orchestrates open-source spatial perception tools.

Key results

• Gemini-3-Pro achieves the top overall SpatialScore of 60.12, while Qwen3-VL-235B-A22B leads open-source models at 56.63, versus human-level 86.60.
• SFT of Qwen3-VL-4B on SpatialCorpus raises overall accuracy from 42.52 to 52.99 (+10.47), and Qwen3-VL-8B from 45.48 to 54.71 (+9.23).
• Training-free SpatialAgent-ReAct lifts Qwen3-VL-8B to 53.81 (+8.33) and Qwen3-VL-4B to 50.30 (+7.78) without any training.

Surprising findings

• Most models fine-tuned on spatial-specific data (e.g., SpaceThinker, SpatialThinker, SpatialReasoner) yield only marginal gains and sometimes underperform their Qwen2.5-VL base models.
• SFT on SpatialCorpus causes drops on some tasks such as view reasoning (e.g., 4B view reasoning falls from 34.75 to 33.86), which the authors attribute to catastrophic forgetting.

Limitations

• Relies primarily on RGB frames; no point cloud, depth map, or normal inputs.
• SpatialCorpus diversity is limited, leading to biased gains across tasks.
• SpatialAgent toolbox is relatively rudimentary and tool errors propagate to answers.

Experimental setup

Datasets: ScanNet++ Omni3D WildRGB-D PointOdyssey CA-1M SRBench SpatialEval SpatialViz-Bench SpatialSense VSR SpatialBench QSpatialBench

Benchmarks: SpatialScore

Metrics: accuracy Mean Relative Accuracy (MRA)

Baselines: GPT-5 Gemini-2.5-Pro Gemini-3-Pro Claude-4.5-Sonnet Qwen3-VL-235B-A22B VST-7B-RL Spatial-MLLM-4B SpaceR-7B SpatialThinker-7B

Builds on: ReAct Reflexion VSI-Bench Omni3D

Closest papers by meaning

• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.93)
• MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.93)
• Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.93)
• SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.93)
• Scaling Spatial Intelligence with Multimodal Foundation Models (CVPR 2026 · similarity 0.92)
• EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.92)