Research map / Vision-language and multimodal / Multimodal LLMs
MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MediConfusion, a 352-question radiology VQA benchmark built from BiomedCLIP-confusing but DINOv2-dissimilar ROCO image pairs, showing that medical and general MLLMs perform at or below random guessing.
Problem
Medical MLLMs may be confused by visually distinct radiology image pairs that their vision encoders map to similar embeddings, and existing medical VQA benchmarks do not probe such reliability failure modes.
Approach
Mine image pairs that are similar in BiomedCLIP embedding space but dissimilar in DINOv2 space, generate paired two-option VQA questions with GPT-4 whose answers differ across the pair, and have radiologists filter them, so language priors alone cannot exceed 50% set-level accuracy.
Key results
- Almost all evaluated models score below random guessing (25% set accuracy); best set accuracy is Gemini 2.0 Flash at 29.55%, with 61.93% individual accuracy.
- Confusion scores are high for most models (often above 90%), indicating models pick the same answer for both images in a pair.
- Medical MLLMs (LLaVA-Med, RadFM, Med-Flamingo) do not outperform general-domain models; LLaVA-Med gets 0.00% set accuracy under MC prompting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate models on visually confusing medical image pairs | MLLMs in the MediConfusion benchmark · MediConfusion | set accuracy (%): most models below 25.00; Gemini 2.0 Flash 29.55 | Hurt |
| Use Gemini 2.0 Flash instead of the other evaluated models | MLLMs · MediConfusion | individual accuracy (%): 61.93 vs 50.00 | Helped |
| Fine-tune LLaVA-Med on the MediConfusion training split (ablation) | LLaVA-Med · MediConfusion test split | set accuracy (%): 0% to around 20% | Helped |
| Fine-tune LLaVA-Med on MediConfusion training data (ablation) | LLaVA-Med · MediConfusion | set accuracy (%) | Hurt |
| Replace confusing image pairs with easy pairs (low BiomedCLIP similarity) (ablation) | GPT-4o · MediConfusion-Easy | set accuracy (%): 18.75 -> 98.38 | Helped |
| Replace confusing image pairs with easy pairs (low BiomedCLIP similarity) (ablation) | LLaVA-Med · MediConfusion-Easy | set accuracy (%): 0.00 -> 34.21 | Helped |
| Remove radiologist feedback from the benchmark curation (ablation) | GPT-4o · MediConfusion-Unfiltered vs MediConfusion | set accuracy (%): 18.75 -> 18.01 | Hurt |
| Remove radiologist feedback from the benchmark curation (ablation) | LLaVA-Med · MediConfusion-Unfiltered vs MediConfusion | set accuracy (%): 0.00 -> 4.86 | Helped |
| Vary the evaluation prompt (ablation) | GPT-4o · MediConfusion | set accuracy (%): 17.33–20.40; All Prompts 19.12 ± 0.04 | No clear effect |
| Vary the evaluation prompt (ablation) | InstructBLIP · MediConfusion | set accuracy (%): 1.25–12.05; All Prompts 4.79 ± 0.12 | Mixed |
| Vary the evaluation prompt (ablation) | LLaVA-Med · MediConfusion | set accuracy (%): 0.00 across prompts | No clear effect |
Surprising findings
- Medical MLLMs trained on the PMC-OA image-caption pairs underlying the benchmark still perform below random guessing.
- On the easy variant, GPT-4o reaches 98.38% set accuracy, showing the failures depend on the difficulty of the confusing pairs.
Limitations
- Benchmark is small (352 questions) and relies on two-option questions from a single dataset (ROCO).
- Question generation and failure-pattern extraction rely on GPT-4/GPT-4o, introducing possible LLM-generated artifacts.
- Results depend on evaluation technique (MC, GD, FF, PS), with proprietary models only evaluated via MC.
Experimental setup
Datasets: ROCO PMC-OA PMC-VQA
Benchmarks: MediConfusion
Metrics: set accuracy individual accuracy confusion score
Baselines: LLaVA BLIP-2 InstructBLIP DeepSeek-VL2 Molmo LLaVA-Med RadFM Med-Flamingo GPT-4o o1 Claude 3 Opus Gemini 1.5 Pro
Builds on: ROCO BiomedCLIP DINOv2 CLIP-blind pairs
Closest papers by meaning
- MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence (NeurIPS 2026 · similarity 0.92)
- OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM (CVPR 2024 · similarity 0.90)
- MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.90)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.90)
- GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI (NeurIPS 2024 · similarity 0.90)
- How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.