Research map / Vision-language and multimodal / Multimodal LLMs
STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?
Yun Li, Yiming Zhang, Tao Lin, Xiangrui Liu, Wenxiao Cai, Zheng Liu, Bo Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Gemini-2.5-Pro instead of GPT-4o | MLLM · STI-Bench | accuracy: 34.8 -> 41.4 | Improved |
| use larger MLLM (Qwen2.5-VL-72B) instead of MiniCPM-V-2.6 | open-source MLLM · STI-Bench | accuracy: 26.9 -> 40.7 | Improved |
| evaluate outdoor instead of desktop scenes | Gemini-2.5-Pro · STI-Bench | accuracy: 35.8 -> 48.7 | Improved |
| evaluate outdoor instead of desktop scenes | Qwen2.5-VL-72B · STI-Bench | accuracy: 33.5 -> 50.6 | Improved |
| precise quantitative task (dimensional measurement) vs pose estimation | Gemini-2.5-Pro · STI-Bench | accuracy: 38.7 vs 50.4 | Hurt |
| displacement and path length vs spatial relation | Gemini-2.5-Pro · STI-Bench | accuracy: 33.9 vs 53.8 | Hurt |
| open-source Qwen2.5-VL-72B vs proprietary Gemini-2.5-Pro | MLLM · STI-Bench | accuracy: 40.7 vs 41.4 | No clear effect |
| ego-centric orientation: Gemini-2.0-Flash vs Gemini-2.5-Pro | Gemini · STI-Bench Ego Orient. | accuracy: 10.8 -> 52.5 | Improved |
| best model vs random baseline | Gemini-2.5-Pro · STI-Bench | accuracy: 20 -> 41.4 | Improved |
Closest papers by meaning
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.91)
- OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding (NeurIPS 2025 · similarity 0.91)
- Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.90)
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.90)
- GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
- Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
