Research map / Vision-language and multimodal / Multimodal LLMs

MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MedAtlas, a multi-round, multi-image, multi-modal medical VQA benchmark from public radiology cases, with Stage Chain Accuracy and Error Propagation Suppression Coefficient metrics, and evaluates 11 general VLMs.

Problem

Existing medical multimodal benchmarks are mostly single-image, single-turn, and do not capture longitudinal, multi-image, multi-modal clinical diagnostic reasoning.

Approach

A staged benchmark of 804 real-world radiology cases where each case unfolds over rounds of imaging and dependent questions, scored with chain-based metrics (SCA) and a cross-round error propagation metric (EPSC) alongside standard QA and findings/diagnosis metrics.

Key results

• Claude-sonnet-4 achieves the highest Knowledge Acc. (0.95), followed by Llama-4-Maverick (0.91) and GPT-4o (0.90).
• Open QA accuracy is much lower than Closed QA across models (e.g., Claude-sonnet-4 36.73 vs 75.76).
• Diagnosis accuracy ranges from 6.5% (Qwen2.5-VL-7B) to 36.6% (Claude-sonnet-4), with Multi-Image Findings correlating with diagnosis (Pearson r = 0.82).

Surprising findings

• Smaller or open models such as InternVL3-78B and Qwen2.5-VL-7B get near-zero Knowledge Acc. despite strong general benchmarks.
• Specialized medical VLM LLaVA-Med was not evaluated in the results because it reportedly failed to follow instructions and supports only single-image inputs.

Limitations

• Cases are drawn from online radiology education sources, which over-represent unusual findings.
• Many questions are multiple choice (72% single-choice, 28% true/false), limiting open-ended evaluation.
• Paper has internal inconsistencies in metric naming (SCA defined as Stage-Chain Accuracy vs Stage Chain Accuracy) and in the reported model results, and no release statement is given.

Experimental setup

Datasets: MedAtlas

Benchmarks: MedAtlas

Metrics: accuracy Stage Chain Accuracy Error Propagation Suppression Coefficient BERTScore ROUGE-L BLEU-1 METEOR Knowledge Acc.

Baselines: GPT-4o Claude-sonnet-4 Claude-3.7 Qwen2.5-VL-72B Qwen-VL-Max DeepSeek V3 InternVL3-78B Llama-4-Maverick Kimi-latest LLaVA-Med

Builds on: VQA-RAD SLAKE PMC-VQA OmniMedVQA Mantis-Eval MMDU

Closest papers by meaning

• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.93)
• Medical thinking with multiple images (ICLR 2026 · similarity 0.91)
• MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding (ICML 2025 · similarity 0.91)
• MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosis (CVPR 2026 · similarity 0.90)
• Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning (ICML 2025 · similarity 0.90)
• Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks (ICLR 2026 · similarity 0.90)