Research map / Vision-language and multimodal / Multimodal LLMs
SpaCE-Eval: A Benchmark for Real-World Multi-Modal Reasoning
Xuyou Yang, Yucheng Zhao, Wenxuan Zhang, Immanuel Koh
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SpaCE-Eval, a benchmark for evaluating multi-modal large language models on real-world multi-modal reasoning (ICLR 2026); full text unavailable, so details are limited to the abstract.
Problem
Existing evaluations of multi-modal LLMs often fail to comprehensively assess their understanding and reasoning in real-world environments, which is a prerequisite for many real-world applications.
Approach
A benchmark aimed at evaluating MLLM understanding and reasoning in real-world multi-modal settings; the specific design is not stated in the available abstract.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale model size up within Gemma-3 family (ablation) | gemma-3 · SpaCE-Eval overall | accuracy: 36.00 -> 37.05 -> 39.77 | Improved |
| scale Qwen2.5-VL from 7B to 72B (ablation) | Qwen2.5-VL · SpaCE-Eval overall | accuracy: 37.14 -> 37.84 | Improved |
| use purely visual options instead of text options (ablation) | MLLMs · SpaCE-Eval | accuracy | Hurt |
| remove image input (ablation) | MLLMs · SpaCE-Eval | accuracy | Hurt |
| increase spatial scale from object to room, building, urban (ablation) | MLLMs · SpaCE-Eval | accuracy | Hurt |
| compare Spatial Reasoning against Commonsense Knowledge category | best model GPT-5 · SpaCE-Eval | accuracy: 66.08 -> 42.25 | Hurt |
| location-view association questions | MLLMs · SpaCE-Eval Spatial Reasoning | accuracy: fewer than 22.73% correct | Hurt |
Limitations
- Full text unavailable; only the abstract was reviewed
Experimental setup
Benchmarks: SpaCE-Eval
Closest papers by meaning
- SpaCE-10: A Comprehensive Benchmark for Multimodal Large Language Models in Compositional Spatial Intelligence (ICLR 2026 · similarity 0.91)
- SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence (CVPR 2026 · similarity 0.90)
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.90)
- From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs (CVPR 2026 · similarity 0.90)
- Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.90)
- SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
