Research map / Vision-language and multimodal / Multimodal LLMs

Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Med-CMR, a 20,653-item medical VQA benchmark decomposed into seven visual and reasoning complexity dimensions, and evaluates 18 MLLMs with MCQ accuracy and LLM-judged open-ended scores.

Problem

Existing medical multimodal benchmarks are dominated by perception-level VQA and do not evaluate complex, multi-step medical reasoning with fine-grained diagnosis of capability gaps.

Approach

Decomposes medical multimodal complexity into three visual dimensions (small-object detection, fine-detail discrimination, spatial understanding) and four reasoning dimensions (temporal prediction, causal reasoning, long-tail generalization, multi-source integration), with template-based question generation from journal case captions, model-assisted filtering and a weighted LLM-judged open-ended s

Key results

• GPT-5 achieves 57.81% MCQ accuracy and 48.70 open-ended score, the best among 18 evaluated MLLMs.
• Qwen3-VL-235B-A22B is the top open-source model with 49.34% MCQ accuracy and 42.62 open-ended score.
• Medical fine-tuned models (Medgemma, Lingshu) do not reliably outperform their general base models on MCQs; Long-tail generalization is the hardest category with top score 55.19%.

Surprising findings

• Medically fine-tuned models such as Medgemma and Lingshu underperform their general base models on MCQ tasks, with the decline attributed to degraded general multimodal reasoning.
• Scaling improves MCQ accuracy but gains in open-ended visual accuracy and ground-truth correctness are much weaker than gains in coherence.

Limitations

• Open-ended scoring depends on a single LLM judge (DeepSeek-V3.2-Exp) and human alignment was checked on only 200 samples with three models.
• Questions are template-generated and partly synthetic (GPT-5-mini), with 8% of synthetic content removed after review.
• Model-assisted filtering removed items answered correctly by three medical/general MLLMs, which may bias difficulty toward those models.

Experimental setup

Datasets: journal case reports and clinical research articles (JMCR, NEJM)

Benchmarks: Med-CMR

Metrics: accuracy open-ended weighted score (consistency, coherence, visual ac ROUGE-L edit distance similarity win ratio spearman correlation

Baselines: GPT-5 Gemini-2.5-Pro Qwen3-VL-235B-A22B InternVL3.5-241B-A28B Lingshu-32B Medgemma-27B Qwen2.5-VL-72B

Builds on: MedXpertQA OmniMedVQA PMC-VQA

Closest papers by meaning

• MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.93)
• Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search (NeurIPS 2025 · similarity 0.92)
• Medical thinking with multiple images (ICLR 2026 · similarity 0.91)
• Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning (ICML 2025 · similarity 0.91)
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.91)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding (ICLR 2026 · similarity 0.90)