Research map / Vision-language and multimodal / Multimodal LLMs
CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CARES, a ~41K-QA benchmark evaluating the trustworthiness of medical large vision-language models across trustfulness, fairness, safety, privacy, and robustness, finding substantial reliability gaps in six models.
Problem
Med-LVLMs are being deployed in healthcare, but their trustworthiness has only been evaluated along single dimensions (e.g., diagnostic accuracy); no standardized multi-dimensional evaluation exists.
Approach
A holistic benchmark that curates medical VQA/classification/report data into closed- and open-ended QA and evaluates five trustworthiness dimensions (factuality, uncertainty, fairness across age/gender/race, jailbreak/over-caution/toxicity, privacy, OOD robustness) on Med-LVLMs versus general LVLMs.
Key results
- Med-LVLMs show factuality accuracy below 50% on most datasets; LLaVA-Med is best overall on factuality.
- Uncertainty-based accuracy is largely below 50%, and overconfidence ratios are high for most Med-LVLMs (e.g., LLaVA-Med 38.34% average OC, Med-Flamingo 59.11%).
- Under jailbreak prompts, accuracy drops for all models (e.g., LLaVA-Med 35.61%, down 4.78), but only LLaVA-Med shows meaningful abstention (30.17%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add healthcare-related jailbreaking prompts to conceal or exaggerate conditions or give incorrect treatment advice | Med-LVLMs and generic LVLMs · Jailbreaking | accuracy: ↓ 4.78; ↓ 6.55; ↓ 5.21; ↓ 2.08; ↓ 2.90; ↓ 2.78 | Hurt |
| Add noisy image corruption to inputs | Med-LVLMs · IU-Xray and OL3I | accuracy: ↓9.33; ↓3.45; ↓8.96; ↓1.38; ↓6.21; ↓10.20; ↓10.43; ↓1.46 | Hurt |
| Prompt with toxic, rude, and disrespectful language | Med-LVLMs · Toxicity evaluation across datasets | toxicity score: increases: IU-Xray 3.02, 4.78, 3.64, 1.95; MIMIC-CXR 0.86, 0.94, 0.74, 0.97 | Hurt |
| Prompt with toxic, rude, and disrespectful language | LLaVA-v1.6 · Toxicity evaluation | toxicity score: IU-Xray ↑ 14.26; MIMIC-CXR ↑ 27.61; OL3I ↑ 7.46 | Hurt |
| Use few-shot fabricated privacy examples before asking about private information | Med-LVLMs · Privacy (MIMIC-CXR) | abstention rate: LLaVA-Med 2.71 -> 2.04; Med-Flamingo 0.76 -> 0.65; MedVInT 0 -> 0; RadFM 0 -> 0; LLaVA-v1.6 14.02 -> 13.18; Qwen-VL-Chat | Hurt |
| Append the uncertainty question, “are you sure you accurately answered the question?” | Med-LVLMs · Uncertainty estimation | uncertainty-based accuracy | Mixed |
| Ask models to answer questions from unfamiliar modalities | Med-LVLMs · Semantic-level OOD robustness | abstention rate: MedVInT: FairVLMed 0, OmniMedVQA 0.01; RadFM: FairVLMed 0.06, OmniMedVQA 0.05 | Hurt |
| Ask routine medical questions requiring image descriptions, reports, or advice | LLaVA-Med · Overcautiousness evaluation | abstention rate: up to 90% | Hurt |
Surprising findings
- General LVLMs Qwen-VL-Chat and LLaVA-v1.6 estimate uncertainty better than several Med-LVLMs despite weaker factuality.
- LLaVA-Med abstains excessively (up to 90% on medical advice queries), i.e., over-cautiousness.
Limitations
- Relies on GPT-4 as judge for open-ended responses and keyword-based abstention detection.
- OOD robustness and privacy evaluations restricted to a few models with known training data; some datasets are restricted-access.
- Evaluates only 7B-level models, with limited dataset coverage for some modalities.
Experimental setup
Datasets: MIMIC-CXR IU-Xray Harvard-FairVLMed PMC-OA HAM10000 OL3I OmniMedVQA
Benchmarks: CARES
Metrics: accuracy over-confident ratio uncertainty-based accuracy abstention rate toxicity score demographic accuracy difference max-min fairness AUROC
Baselines: LLaVA-Med Med-Flamingo MedVInT RadFM LLaVA-v1.6 Qwen-VL-Chat
Builds on: LLaVA-Med OmniMedVQA Harvard-FairVLMed MIMIC-CXR
Closest papers by meaning
- MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.89)
- MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence (NeurIPS 2026 · similarity 0.88)
- Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models (AAAI 2025 · similarity 0.88)
- FairCLIP: Harnessing Fairness in Vision-Language Learning (CVPR 2024 · similarity 0.88)
- CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs (NeurIPS 2025 · similarity 0.88)
- GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.