AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

SpaCE-Eval: A Benchmark for Real-World Multi-Modal Reasoning

Xuyou Yang, Yucheng Zhao, Wenxuan Zhang, Immanuel Koh

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SpaCE-Eval, a benchmark for evaluating multi-modal large language models on real-world multi-modal reasoning (ICLR 2026); full text unavailable, so details are limited to the abstract.

Problem

Existing evaluations of multi-modal LLMs often fail to comprehensively assess their understanding and reasoning in real-world environments, which is a prerequisite for many real-world applications.

Approach

A benchmark aimed at evaluating MLLM understanding and reasoning in real-world multi-modal settings; the specific design is not stated in the available abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
scale model size up within Gemma-3 family (ablation)gemma-3 · SpaCE-Eval overallaccuracy: 36.00 -> 37.05 -> 39.77Improved
scale Qwen2.5-VL from 7B to 72B (ablation)Qwen2.5-VL · SpaCE-Eval overallaccuracy: 37.14 -> 37.84Improved
use purely visual options instead of text options (ablation)MLLMs · SpaCE-EvalaccuracyHurt
remove image input (ablation)MLLMs · SpaCE-EvalaccuracyHurt
increase spatial scale from object to room, building, urban (ablation)MLLMs · SpaCE-EvalaccuracyHurt
compare Spatial Reasoning against Commonsense Knowledge categorybest model GPT-5 · SpaCE-Evalaccuracy: 66.08 -> 42.25Hurt
location-view association questionsMLLMs · SpaCE-Eval Spatial Reasoningaccuracy: fewer than 22.73% correctHurt

Limitations

Experimental setup

Benchmarks: SpaCE-Eval

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy