Research map / Vision-language and multimodal / Multimodal LLMs
SITE: towards Spatial Intelligence Thorough Evaluation
Wenqi Wang, Reuben Tan, Pengyue Zhu, Jianwei Yang, Zhengyuan Yang, Lijuan Wang, Andrey Kolobov, Jianfeng Gao, Boqing Gong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale LLAVA-OneVision from 0.5B to 7B | LLAVA-OneVision · SITE full | chance-adjusted accuracy: 18.4 -> 30.2 | Improved |
| scale Qwen2.5-VL from 3B to 7B | Qwen2.5-VL · SITE full | chance-adjusted accuracy: 29.5 -> 31.4 | Improved |
| scale InternVL-2.5 from 4B to 8B | InternVL-2.5 · SITE full | chance-adjusted accuracy: 29.4 -> 32.8 | Improved |
| compare small open-source Qwen2.5-VL-7B and InternVL-2.5-8B with proprietary GPT-4o | open-source VLMs · SITE full | chance-adjusted accuracy: 31.4 and 32.8 vs 37.8 | No clear effect |
| use ego-exo view association instead of standard tasks | GPT-4o · SITE view association | chance-adjusted accuracy: 35.70 and 20.70 vs human 100 | Hurt |
| use frames reordering instead of view association | GPT-4o · SITE new tasks | chance-adjusted accuracy: 35.70 -> -2.01 (ego2exo) | Hurt |
| compare Qwen2.5-VL-7B with GPT-4o on frames reordering | Qwen2.5-VL-7B · SITE frames reordering | chance-adjusted accuracy: 7.63 vs -2.01 (ego2exo) | Improved |
| evaluate multi-view reasoning on full split | state-of-the-art VLMs · SITE Multi-View | chance-adjusted accuracy: 22.0 (GPT-4o) vs 87.5 (human, tiny) | Hurt |
| fine-tune Qwen2.5-VL-7B instead of smaller VLMs on LIBERO-Spatial | VLM backbone · LIBERO-Spatial | success rate (%): 0.0 -> 38.0 | Improved |
| correlate SITE score with robot manipulation performance | SITE · LIBERO-Spatial | Pearson correlation: 0.902 | Improved |
| correlate POPE score with robot manipulation performance | POPE · LIBERO-Spatial | Pearson correlation: -0.602 | Hurt |
| correlate MathVista score with robot manipulation performance | MathVista · LIBERO-Spatial | Pearson correlation: 0.935 | Improved |
Closest papers by meaning
- SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.92)
- Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.92)
- Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests (NeurIPS 2026 · similarity 0.91)
- Scaling Spatial Intelligence with Multimodal Foundation Models (CVPR 2026 · similarity 0.91)
- MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence (ICLR 2026 · similarity 0.90)
- Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
