Research map / Vision-language and multimodal / Multimodal LLMs
MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
Huanjin Yao, Jiaxing Huang, Yawen Qiu, Michael K. Chen, Wenzheng Liu, Wei Zhang, Wenjie Zeng, Xikun Zhang, Jingyi Zhang, YuXin Song, Wenhao Wu, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove answer choices from the question (ablation) | Qwen2-VL-7B · MathVista | accuracy: 58.2 -> 37.8 | Hurt |
| remove answer choices from the question (ablation) | GPT-4o · MathVista | accuracy: 63.8 -> 40.8 | Hurt |
| apply multi-model voting filtering to candidate questions (ablation) | GPT-4o · MMReason candidates | text-only accuracy: 13.4 -> 0.78 | Improved |
| apply multi-model voting filtering to candidate questions (ablation) | GPT-4o · MMReason candidates | multimodal relevance rate (%): 56.5 -> 97.0 | Improved |
| apply multi-model voting filtering to candidate questions (ablation) | Qwen2.5-VL-7B · MMReason candidates | multimodal relevance rate (%): 60.7 -> 96.1 | Improved |
| apply voting filtering, measured with text and visual input (ablation) | GPT-4o · MMReason candidates | accuracy: 30.8 -> 25.7 | Hurt |
| use newly collected questions instead of reformulated M3CoT questions | GPT-4o-1120 · MMReason | final answer reasoning accuracy: 28.9 -> 21.8 | Hurt |
| scale Qwen-2.5-VL from 7B to 72B | Qwen-2.5-VL · MMReason | final answer reasoning accuracy: 16.8 -> 24.7 | Improved |
| scale InternVL-2.5-MPO from 8B to 78B | InternVL-2.5-MPO · MMReason | final answer reasoning accuracy: 12.4 -> 21.3 | Improved |
| use explicit chain-of-thought training (LLaVA-CoT, Mulberry) over base model | LLaMA-3.2-Vision-11B · MMReason | final answer reasoning accuracy: 11.9 -> 12.5 | No clear effect |
Closest papers by meaning
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.93)
- Think 360deg: Beyond Depth: Evaluating the Width-centric Reasoning Capability of MLLMs (CVPR 2026 · similarity 0.91)
- RBench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation (ICML 2025 · similarity 0.90)
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.90)
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.90)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
