Research map / Vision-language and multimodal / Multimodal LLMs
SpatialLogic-Bench: A Diagnostic Benchmark for Task-Oriented Spatiotemporal Reasoning
Xiaoda Yang, Shenzhou Gao, Can Wang, Jiahe Zhang, Menglan Tang, Jingyang Xue, Sheng Liu, Peijian Zhang, Yao Mu, Xiangyu Yue
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SpatialLogic-Bench, a benchmark for evaluating vision-language models on task-oriented spatiotemporal reasoning in dynamic scenes.
Problem
Existing benchmarks for vision-language models focus on static spatial relationships and do not systematically assess dynamic, task-oriented spatiotemporal reasoning.
Approach
A diagnostic benchmark targeting spatiotemporal logic in VLMs, aimed at dynamic task-oriented reasoning rather than static perception.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ground-truth state-change analysis to prompt (ablation) | GPT-4o · inverse task (window >=12) | accuracy (%): 58.3 -> 95.2 | Improved |
| add ground-truth state-change analysis to prompt (ablation) | GPT-o4mini · inverse task (window >=12) | accuracy (%): 40.5 -> 98.6 | Improved |
| add ground-truth state-change analysis to prompt (ablation) | Paligemma · inverse task (window >=12) | accuracy (%): 8.3 -> 12.2 | No clear effect |
| add ground-truth state-change analysis to prompt (ablation) | InternVL-8B · inverse task (window >=12) | accuracy (%): 15.8 -> 21.5 | No clear effect |
| add ground-truth state-change analysis to prompt (ablation) | DeepSeek-VL2 · inverse task (window >=12) | accuracy (%): 69.0 -> 20.6 | Hurt |
| add ground-truth state-change analysis to prompt (ablation) | Video-3D LLM · inverse task (window >=12) | accuracy (%): 67.4 -> 78.5 | Improved |
| reverse frame order (inverse vs forward task) | GPT-4o · SpatialLogic-Bench | accuracy (%): 99.8 -> 53.7 | Hurt |
| reverse frame order (inverse vs forward task) | Paligemma · SpatialLogic-Bench | accuracy (%) | Hurt |
| reverse frame order (inverse vs forward task) | Gemini-2.5-Pro · SpatialLogic-Bench | accuracy (%): 82.7 -> 77.7 | No clear effect |
| increase window size between frames | most VLMs · SpatialLogic-Bench | accuracy (%): Gemini-2.5-Pro 71.3 -> 100.0 | Improved |
| increase window size between frames | Qwen2.5-VL-7B · SpatialLogic-Bench | accuracy (%): 53.1 -> 56.5 | No clear effect |
| use 2D open-source model instead of 3D-aware | open-source VLMs · SpatialLogic-Bench | accuracy (%): 3D-LLM 27.0 vs DeepSeek-VL2 64.4 (window 5) | Improved |
Experimental setup
Benchmarks: SpatialLogic-Bench
Closest papers by meaning
- Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (ICLR 2026 · similarity 0.92)
- Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
- SpatiaLab: Can Vision–Language Models Perform Spatial Reasoning in the Wild? (ICLR 2026 · similarity 0.91)
- Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D World (CVPR 2026 · similarity 0.91)
- OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding (NeurIPS 2025 · similarity 0.91)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
