Research map / Vision-language and multimodal / Multimodal LLMs

MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MMR-Life, a 2,646-question multiple-choice benchmark of real-life multi-image reasoning across seven reasoning types, and evaluates 37 MLLMs and their reasoning paradigms.

Problem

Existing multimodal reasoning benchmarks rely on expert knowledge or symbolic single-image puzzles, and do not evaluate reasoning over multiple real-life images across diverse reasoning types.

Approach

A benchmark of real-life natural multi-image inputs (avg 7.22 images per question) spanning seven reasoning types and 21 tasks, requiring only commonsense rather than domain expertise, with multiple-choice answers and difficulty/format/quality filtering; it also analyzes thinking length, reasoning enhancement methods (CoT, SC, BoN, GRPO), and correlations among reasoning types.

Key results

• GPT-5 achieves 58.69% average accuracy on MMR-Life, versus 72.28% for human participants on the mini set.
• Spatial reasoning is hardest: the best model reaches only 25.10% (Gemini-2.5-Pro), against 79.76% for humans.
• On Qwen2.5-VL-72B, BoN@8 (39.80) and GRPO (39.68) fall below CoT (40.21), while SC@8 is marginally higher (40.33).

Surprising findings

• Open-source thinking models on average (27.15%) do not outperform open-source non-thinking models (29.01%).
• Several models, such as Skywork-R1V-38B and InternVL3.5-8B, score below random guessing (20%).

Limitations

• Human baseline uses only a 210-question mini set with 12 students.
• Error analysis covers only GPT-5 and Gemini-2.5-Pro on 140 sampled errors.

Experimental setup

Datasets: TVBench Tom & Jerry Cartoon CLEVRER-Humans Kaggle image datasets eBird ContPhy MathVista RecipeQA ViewSpatial-Bench NAVI Drivingdojo

Benchmarks: MMR-Life

Metrics: accuracy

Baselines: GPT-5 Gemini-2.5-Pro Claude-Sonnet-4 o4-mini Qwen2.5-VL-72B VL-Rethinker-72B MM-Eureka-Qwen-32B MiMo-VL-7B-RL

Builds on: chain-of-thought GRPO Best-of-N Self-Consistency

Closest papers by meaning

• LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.94)
• MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos (ICLR 2026 · similarity 0.93)
• Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark (ICML 2025 · similarity 0.92)
• MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.92)
• MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs (NeurIPS 2024 · similarity 0.91)
• RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation (AAAI 2026 · similarity 0.91)