Research map / Vision-language and multimodal / Multimodal LLMs

Medical thinking with multiple images

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MedThinkVQA, an expert-annotated radiology benchmark of 8,067 multi-image Eurorad cases (6.62 images/case) with per-image findings, integrated summaries, and stepwise/beyond-accuracy evaluation, showing top VLMs reach only ~57% accuracy.

Problem

Medical VQA benchmarks mostly use single images and answer-only scoring, whereas real diagnosis requires grounding, aligning, and composing evidence across multiple images; existing benchmarks lack multi-image density and stepwise supervision.

Approach

A multi-image diagnostic benchmark built from expert-annotated Eurorad cases with a three-step think-with-images supervision (per-image findings, case-level integrated summary, differential-diagnosis reasoning), filtered against text-solvable shortcuts and option-length bias, plus step-level LLM-judge and human evaluation.

Key results

• Best test accuracy is Claude-4.6-Opus at 57.2%, followed by Gemini-3-Pro at 55.3% and GPT-5.2 (xhigh) at 54.9%; top open model Qwen3.5-397B-A17B reaches 52.2%.
• Adding expert integrated imaging summaries raises accuracy by +41.5 to +50.5 points across MedGemma-27B and GPT-5 family models, while self-generated hints/summaries consistently reduce accuracy.
• Over 70% of step errors are image-understanding errors (69.23% on critical error-bearing steps); human expert accuracy on a 96-case subset is 77.10%.

Surprising findings

• Self-generated image descriptions or summaries consistently hurt accuracy across all tested models, despite expert versions giving large gains.
• Extra reasoning tokens help only for models with reliable visual grounding; small Qwen3.5 and Qwen3-VL models show flat or unstable gains, and Qwen3.5-0.8B's reasoning can reiterate misreadings.

Limitations

• Single educational source (Eurorad) with possible distribution shift across hospitals and populations; English-only.
• Test set is 720 cases and LLM-judge components may be prompt-sensitive and carry judge bias; human study covers only 50 cases and 96 cases.
• Multiple-choice setting simplifies real diagnostic work; rare diseases and some specialties underrepresented.

Experimental setup

Datasets: Eurorad

Benchmarks: MedThinkVQA

Metrics: accuracy ROUGE-L RadCliQ-v1 step-level factual correctness error-type coverage rubric score MELD

Baselines: Claude-4.6-Opus Gemini-3-Pro GPT-5.2 GPT-5-mini GPT-5-nano Qwen3.5-397B-A17B Qwen3.5-27B Lingshu-32B InternVL3.5-38B MedGemma-27B

Builds on: Eurorad MedXpertQA chain-of-thought

Closest papers by meaning

• MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.91)
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.91)
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.91)
• M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding (ICLR 2026 · similarity 0.90)
• Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning (ICML 2025 · similarity 0.89)
• CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework (ICLR 2026 · similarity 0.89)