Research map / Vision-language and multimodal / Multimodal LLMs
VSP: Diagnosing the Dual Challenges of Perception and Reasoning in Spatial Planning Tasks for MLLMs
Qiucheng Wu, Handong Zhao, Michael Saxon, Trung Bui, William Yang Wang, Yang Zhang, Shiyu Chang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| LoRA fine-tune on 10k VSP examples | LLaVA-v1.6-Vicuna-7B · Maze Navigation main task | success rate: 0.05 -> 0.60 | Improved |
| LoRA fine-tune on 10k VSP examples | LLaVA-v1.6-Vicuna-7B · Blocks World main task | success rate: 0.01 -> 0.97 | Improved |
| LoRA fine-tune on 10k VSP examples | InternLM-XComposer2-7B · Maze Navigation main task | success rate: 0.11 -> 0.17 | Improved |
| LoRA fine-tune on 10k VSP examples | InternLM-XComposer2-7B · Blocks World main task | success rate: 0.00 -> 0.09 | Improved |
| add 4 in-context examples | GPT-4V · Maze Navigation T2 | accuracy: 0.27 -> 0.69 | Improved |
| add 2 in-context examples | Gemini-1.0 · Blocks World T3 | accuracy: 0.54 -> 0.76 | Improved |
| add 4 in-context examples | Gemini-1.0 · Maze Navigation main task | success rate: 0.17 -> 0.19 | No clear effect |
| add 4 in-context examples | GPT-4V · Maze Navigation T1 | accuracy: 0.56 -> 0.54 | No clear effect |
| add 4 in-context examples | Gemini-1.0 · Blocks World main task | success rate: 0.03 -> 0.06 | No clear effect |
| increase maze size from 3x3 to 8x8 | GPT-4V · Maze Navigation main task | success rate: 0.55 -> 0.10 | Hurt |
| replace image input with textual description of environment (ablation) | Gemini · Maze Navigation and Blocks World tasks | accuracy | Improved |
| upgrade Gemini-1.0 to Gemini-2.0 | Gemini · VSP overall | success rate: 0.1167 -> 0.4533 | Improved |
Closest papers by meaning
- Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.92)
- MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.92)
- Visual Planning: Let's Think Only with Images (ICLR 2026 · similarity 0.92)
- SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.92)
- Evaluating Vision-Language Models as Evaluators in Path Planning (CVPR 2025 · similarity 0.91)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
