Research map / Vision-language and multimodal / Multimodal LLMs
Spatial reasoning in VLMs
Improves grounded spatial reasoning and perspective understanding in vision-language models.
207 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Multimodal LLMs, keyword cluster “vlms · visual reasoning · chain”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
7 papers in 2023–24 and 198 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 3.4% to 9.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · Qualitative Spatial Logic over 2D Euclidean Spaces Is Not Finitely Axiomatisable (AAAI 2019 · 0 citations)
- 2020 · Region-Based Global Reasoning Networks (AAAI 2020 · 4 citations)
- 2024 · SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities (CVPR 2024 · 234 citations)
- 2025 · Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (CVPR 2025 · 92 citations)
- 2026 · RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow (AAAI 2026 · 4 citations)
Earliest papers here
- Qualitative Spatial Logic over 2D Euclidean Spaces Is Not Finitely Axiomatisable (AAAI 2019 · 0 citations)
- Region-Based Global Reasoning Networks (AAAI 2020 · 4 citations)
- SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities (CVPR 2024 · 234 citations)
What has been tried
53 papers here report 592 measured changes (a component added, removed or swapped, compared with a baseline): helped 75%, no clear effect 4%, hurt 19%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Unfreeze the ViT encoder during finetuning on spatial data | Manipulation quantitative VQA · In range [50,200]% | 34.9% to 37.2% (+2.3) | Helped | SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities (CVPR 2024 · 234 citations) |
| add relative-depth inputs to the RGB model | SpatialRGPT-Bench qualitative average · success rate | 89.80 -> 91.78 | Helped | SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · 98 citations) |
| Use a ground-truth cognitive map instead of the model-predicted map | VSI-Bench relative distance · accuracy | 56.0 -> 66.0 | Helped | Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces (CVPR 2025 · 92 citations) |
| Remove lane perception supervision | nuScenes open-loop planning · Collision (%) | 1.90 -> 4.65 | Hurt | OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning (CVPR 2025 · 34 citations) |
| Add multi-stage pre-training | GeoChat-Instruct · mAP@0.5 (multiple referred objects) | +5% | Helped | EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues (CVPR 2025 · 32 citations) |
| Increase the number of spatial relationship annotations | RoboSpatial-Val · Average accuracy | 38.1 -> 46.7 -> 60.5 -> 65.8 -> 72.4 | Helped | RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics (CVPR 2025 · 30 citations) |
| Use integer-valued coordinate binning instead of normalized floating point coordinates | GQA · accuracy | 46.1 -> 47.3 | Helped | Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs (CVPR 2024 · 28 citations) |
| Add visual image input | 11 room types · PSA | 48.2 ± 1.5 -> 58.8 ± 3.4 | Helped | LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models (CVPR 2025 · 27 citations) |
| Add textual descriptions to vision-only input | Spatial-Map and Spatial-Grid · accuracy | VQA to VTQA | Helped | Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · 23 citations) |
| Add interleaved text-image input instead of text-only input | MSQA · correctness score | 56.48 -> 54.13 | Hurt | Multi-modal Situated Reasoning in 3D Scenes (NeurIPS 2024 · 11 citations) |
| Add 2D data to the SFT recipe | BLINK_val · accuracy | 74.48 -> 89.27 | Helped | RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics (NeurIPS 2025 · 9 citations) |
| Use SwinT instead of ResNet-50 | Camera-only corruption benchmark · mCE | 100.0 -> 77.8 | Helped | Is Your HD Map Constructor Reliable under Sensor Corruptions? (NeurIPS 2024 · 8 citations) |
What reviewers will expect
Datasets and baselines named in the 198 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ScanNet · 24 of 198 papers
- ARKitScenes · 14 of 198 papers
- VSI-Bench · 13 of 198 papers
- nuScenes · 10 of 198 papers
- SQA3D · 10 of 198 papers
- Matterport3D · 9 of 198 papers
- MindCube · 9 of 198 papers
- CV-Bench · 8 of 198 papers
Baselines
- GPT-4o · 68 of 198 papers
- Gemini-2.5-Pro · 45 of 198 papers
- GPT-5 · 22 of 198 papers
- Qwen2.5-VL-72B · 22 of 198 papers
- Qwen2.5-VL-7B · 19 of 198 papers
- Gemini-1.5-Pro · 17 of 198 papers
- Gemini-2.5-Flash · 17 of 198 papers
- Gemini-2.0-Flash · 15 of 198 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 205 papers; from OpenAlex and the papers' affiliations).
- Shanghai Jiao Tong University · 9 papers
- Google · 8 papers
- Zhejiang University · 7 papers
- Peking University · 7 papers
- NVIDIA · 6 papers
- The University of Hong Kong · 6 papers
- Johns Hopkins University · 5 papers
- Hong Kong University of Science and Technology · 5 papers
- Tsinghua University · 5 papers
- Sun Yat-sen University · 4 papers
Use it
Download BibTeX of 21 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- VLM capability analysis (264 papers)
- Chain-of-thought multimodal reasoning (332 papers)
- Multimodal LLM benchmarks (252 papers)
- Reinforcement learning for visual reasoning (163 papers)
Elsewhere in the topic
- Instruction-tuned multimodal assistants (199 papers)
- Benchmarks and analysis of MLLMs (181 papers)
- Interleaved image-text generation with MLLMs (153 papers)
- Unified any-modality language models (152 papers)
- Attention-based visual token pruning (117 papers)
- Decoding-time hallucination mitigation (91 papers)
- Training-free token reduction for MLLMs (90 papers)
- Visual instruction tuning (68 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).