Research map / Vision-language and multimodal / Multimodal LLMs
MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradations
Jiyao Liu, Junzhi Ning, Chenglong Ma, Wanying Qu, Jianghan Shen, Siqi Luo, Jinjie Wei, Jin Ye, Pengze Li, Tianbin Li, Jiashi Lin, Hongming Shan and 5 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase degradation severity from L0 to L2 | InternVL3-Instruct(78B) · MedQ-Deg overall | accuracy: 74.04 -> 68.63 | Hurt |
| increase degradation severity from L0 to L1 | InternVL3-Instruct(78B) · MedQ-Deg overall | accuracy: 74.04 -> 72.46 | Hurt |
| increase degradation severity from L1 to L2 | GPT-5 · MedQ-Deg overall | accuracy: 69.09 -> 64.30 | Hurt |
| increase degradation severity from L0 to L1 | Hulu-Med(14B) · MedQ-Deg overall | accuracy: 70.04 -> 70.20 | No clear effect |
| increase degradation severity from L0 to L2 | Gemma-3(27B) · MedQ-Deg overall | accuracy: 61.03 -> 52.45 | Hurt |
| apply artifact degradations (vs intensity jitter) | all 40 MLLMs · MedQ-Deg degradation categories | accuracy drop L0 to L1&L2: 5.4 vs 2.0 average drop | Hurt |
| apply motion interference degradations | all 40 MLLMs · MedQ-Deg degradation categories | accuracy drop L0 to L1&L2: 4.6 average drop | Hurt |
| apply noise or resolution & blur degradations | all 40 MLLMs · MedQ-Deg degradation categories | accuracy drop L0 to L1&L2: 3.2 and 3.0 average drop | Mixed |
| evaluate Anatomical Recognition vs Treatment under degradation | all 40 MLLMs · MedQ-Deg capability dimensions | accuracy drop L0 to L1&L2: 4.5 vs 1.8 average drop | Mixed |
| use medical-specialized models instead of general open-source models | medical MLLMs · MedQ-Deg capability average (L1&L2) | accuracy: 35.6 vs 31.6 | Mixed |
| use commercial models instead of open-source models | commercial MLLMs · MedQ-Deg capability average (L1&L2) | accuracy: 49.2 vs 31.6 | Improved |
| increase severity from L0 to L2 | all models · MedQ-Deg calibration shift | calibration shift | Hurt |
Closest papers by meaning
- How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.89)
- MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.89)
- OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks (CVPR 2026 · similarity 0.88)
- MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence (NeurIPS 2026 · similarity 0.88)
- OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM (CVPR 2024 · similarity 0.88)
- MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
