Research map / Vision-language and multimodal / Multimodal LLMs
MMCR: Benchmarking Cross-Source Reasoning in Scientific Papers
Yang Tian, Zheng Lu, Mingqi Gao, Zheng Liu, Bo Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use circular evaluation (5 option rotations) instead of single-pass (ablation) | MMCR evaluation · MMCR | accuracy: MiniCPM-o 2.6 drops to approximately 36% of its non-circular score; random 20% -> 0.2^5 x 100% | Hurt |
| raise rendering DPI from 36 to 108 (ablation) | VLMs on MMCR · MMCR | accuracy: steady improvements as DPI increased to 108 | Improved |
| raise rendering DPI above 108 up to 180 (ablation) | VLMs on MMCR · MMCR | accuracy: performance fluctuates and even declines when DPI exceeds 108 | Hurt |
| add CoT prompt to small VLMs (ablation) | small VLMs (e.g. MiniCPM-o 2.6) · MMCR | accuracy: notable performance decline | Hurt |
| add CoT prompt to large VLMs (ablation) | InternVL2.5-78B and Qwen2.5-VL-72B · MMCR | accuracy: performance improved with CoT prompt | Improved |
| add CoT prompt to Idefics3-8B (ablation) | Idefics3-8B · MMCR | accuracy: slight improvement; model still answered directly | No clear effect |
| proprietary GPT-4o versus best open-source Qwen2.5-VL-72B | GPT-4o · MMCR overall | accuracy: 39.86 -> 48.55 (8.69% advantage) | Improved |
| scale Qwen2.5-VL from 7B to 72B | Qwen2.5-VL · MMCR overall | accuracy: 21.74 -> 39.86 | Improved |
| scale InternVL2.5 from 8B to 78B | InternVL2.5 · MMCR overall | accuracy: 12.32 -> 14.49 | Improved |
| table questions versus figure questions | evaluated VLMs · MMCR | accuracy: 72.22% (13/18) of models equal or worse on table than figure questions | Hurt |
| concatenate multi-page images into one image for models without multi-image support | VLMs marked with * · MMCR overall | accuracy: concatenated models score 0.36-4.71 overall | Hurt |
Closest papers by meaning
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
- MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning (ICLR 2026 · similarity 0.88)
- MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models (AAAI 2026 · similarity 0.88)
- Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning (NeurIPS 2025 · similarity 0.87)
- HoneyBee: Data Recipes for Vision-Language Reasoners (CVPR 2026 · similarity 0.87)
- MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
