AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

MIRA: Evaluating Multimodal AI on Complex Clinical Reasoning in Interventional Radiology

Jingxiong Li, Chenglu Zhu, Sunyi Zheng, Yuxuan Sun, Yifei Wang, He Liu, Yunlong Zhang, Yixuan Si, Lin Yang, Liang Xiao

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MIRA, a benchmark for evaluating large multimodal models on expert-level interventional radiology reasoning with diverse question formats.

Problem

Existing medical multimodal benchmarks mostly use binary labels without contextual depth, so they do not test expert-level interventional radiology knowledge and visual reasoning.

Approach

A benchmark for interventional radiology with diverse question formats (open-ended, closed-form and others) designed to require specialized domain knowledge and advanced visual reasoning rather than binary labels.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
LoRA fine-tune on MIRAQwen2-VL-2B · MIRA single-choiceaccuracy: 0.23 -> 0.71Improved
LoRA fine-tune on MIRALLaVA1.5-13B · MIRA closed-endedaccuracy: 0.50 -> 0.84Improved
LoRA fine-tune on MIRALLaVA-next-8B · MIRA multiple-choiceaccuracy: 0.09 -> 0.32Improved
LoRA fine-tune on MIRAQwen2-VL-7B · MIRA open-endedBLEU-4: 5.73 -> 18.76Improved
provide the image in addition to textGPT4.1-mini / Qwen-plus text-only vs GPT4o · MIRA closed-endedaccuracy: 0.38 -> 0.76Improved
add open-ended question fine-tuning (ablation)LLaVA1.5-7B · MIRA closed-endedBLEU-4: 7.08 -> 15.27Improved
add multiple-choice questions to open-ended fine-tuning (ablation)LLaVA1.5-7B · MIRA closed-endedBLEU-4: 15.27 -> 18.43Improved
add multiple-choice questions to open-ended fine-tuning (ablation)Qwen2-VL-7B · MIRA single-choiceBLEU-4: 7.86 -> 23.95Improved
add multiple-choice questions to open-ended fine-tuning (ablation)Qwen2-VL-7B · MIRA single-choiceROUGE-L: 22.39 -> 37.61Improved
raise IR-term filter threshold from 1 to 6 (ablation)data curation · validation set of 4000precision / recall: precision 74.91% -> 98.92%, recall 94.35% -> 41.1%Mixed
use threshold of two IR terms (ablation)data curation · validation set of 4000precision / recall: precision 90.21%, recall 84.25%Improved
use larger text-only LLM (GPT4.1 vs GPT4.1-mini)text-only LLMs · MIRA closed-endedaccuracy: 0.38 -> 0.14Hurt

Experimental setup

Benchmarks: MIRA

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy