Research map / Vision-language and multimodal / Multimodal LLMs
MIRA: Evaluating Multimodal AI on Complex Clinical Reasoning in Interventional Radiology
Jingxiong Li, Chenglu Zhu, Sunyi Zheng, Yuxuan Sun, Yifei Wang, He Liu, Yunlong Zhang, Yixuan Si, Lin Yang, Liang Xiao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MIRA, a benchmark for evaluating large multimodal models on expert-level interventional radiology reasoning with diverse question formats.
Problem
Existing medical multimodal benchmarks mostly use binary labels without contextual depth, so they do not test expert-level interventional radiology knowledge and visual reasoning.
Approach
A benchmark for interventional radiology with diverse question formats (open-ended, closed-form and others) designed to require specialized domain knowledge and advanced visual reasoning rather than binary labels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| LoRA fine-tune on MIRA | Qwen2-VL-2B · MIRA single-choice | accuracy: 0.23 -> 0.71 | Improved |
| LoRA fine-tune on MIRA | LLaVA1.5-13B · MIRA closed-ended | accuracy: 0.50 -> 0.84 | Improved |
| LoRA fine-tune on MIRA | LLaVA-next-8B · MIRA multiple-choice | accuracy: 0.09 -> 0.32 | Improved |
| LoRA fine-tune on MIRA | Qwen2-VL-7B · MIRA open-ended | BLEU-4: 5.73 -> 18.76 | Improved |
| provide the image in addition to text | GPT4.1-mini / Qwen-plus text-only vs GPT4o · MIRA closed-ended | accuracy: 0.38 -> 0.76 | Improved |
| add open-ended question fine-tuning (ablation) | LLaVA1.5-7B · MIRA closed-ended | BLEU-4: 7.08 -> 15.27 | Improved |
| add multiple-choice questions to open-ended fine-tuning (ablation) | LLaVA1.5-7B · MIRA closed-ended | BLEU-4: 15.27 -> 18.43 | Improved |
| add multiple-choice questions to open-ended fine-tuning (ablation) | Qwen2-VL-7B · MIRA single-choice | BLEU-4: 7.86 -> 23.95 | Improved |
| add multiple-choice questions to open-ended fine-tuning (ablation) | Qwen2-VL-7B · MIRA single-choice | ROUGE-L: 22.39 -> 37.61 | Improved |
| raise IR-term filter threshold from 1 to 6 (ablation) | data curation · validation set of 4000 | precision / recall: precision 74.91% -> 98.92%, recall 94.35% -> 41.1% | Mixed |
| use threshold of two IR terms (ablation) | data curation · validation set of 4000 | precision / recall: precision 90.21%, recall 84.25% | Improved |
| use larger text-only LLM (GPT4.1 vs GPT4.1-mini) | text-only LLMs · MIRA closed-ended | accuracy: 0.38 -> 0.14 | Hurt |
Experimental setup
Benchmarks: MIRA
Closest papers by meaning
- MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.90)
- MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.89)
- AOR: Anatomical Ontology-Guided Reasoning for Medical Large Multimodal Model in Chest X-Ray Interpretation (NeurIPS 2025 · similarity 0.88)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.88)
- Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.87)
- 3D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Tasks (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
