Research map / Vision-language and multimodal / Multimodal LLMs
MedLesionVQA: A Multimodal Benchmark Emulating Clinical Visual Diagnosis for Body Surface Health
Deli Yu, Shengzhi Wang, Kai WU, Xiaozhong Ji, Bo Cui, Jieqiong Cao, Huichao Wang, Boyuan Jiang, Xu Wang, Qian Xu, ChaoGao, Yi Zhao and 5 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MedLesionVQA, a multimodal benchmark for body-surface health conditions that emulates stepwise clinical visual diagnosis, addressing limitations of publicly sourced or disease-classification-only medical benchmarks.
Problem
Existing medical multimodal benchmarks are either built from public sources with limited expert curation or focus narrowly on disease classification, failing to capture the stepwise recognition and reasoning process of clinical visual diagnosis for body-surface conditions.
Approach
A benchmark for body-surface health conditions across clinical departments that is designed to mirror stepwise clinical visual diagnosis (lesion recognition and reasoning) rather than disease classification alone, with expert curation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add image input to text (ablation) | GPT-4V · MedLesionVQA validation | average accuracy: 0.3089 -> 0.4938 | Improved |
| add image input to text (ablation) | Qwen2.5-VL-72B · MedLesionVQA validation | average accuracy: 0.3478 -> 0.4904 | Improved |
| add image input to text (ablation) | GPT-4V · Suggestion & Treatment | accuracy: 0.8140 -> 0.8216 | No clear effect |
| medical-domain fine-tuning (LLaVA-med vs LLaVA-v1.5, 7B) | LLaVA-v1.5-7B · MedLesionVQA validation | average accuracy: 0.2648 -> 0.0885 | Hurt |
| medical-domain model (MedGemma-4b vs InternVL2.5-4B) | 4B MLLM · MedLesionVQA validation | average accuracy: 0.2632 -> 0.195 | Hurt |
| increase model size beyond 10B | open-source MLLMs · MedLesionVQA | average accuracy: GLM-4V-9B 0.4654 vs InternVL2.5-38B 0.3096 | Mixed |
| increase model size from under 1B to 1B-10B | open-source MLLMs · MedLesionVQA | average accuracy: below 0.2 -> 0.3 to 0.4 | Improved |
| compare best MLLM with primary physicians | Gemini-2.5-pro · MedLesionVQA validation | average accuracy: 0.5624 vs 0.6144 | Hurt |
| add color and size rubric to judge prompt (ablation) | GPT-4 judge · open-ended scoring | judge-human consistency | Improved |
Experimental setup
Benchmarks: MedLesionVQA
Closest papers by meaning
- OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM (CVPR 2024 · similarity 0.89)
- Medical thinking with multiple images (ICLR 2026 · similarity 0.89)
- How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.89)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.89)
- MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.88)
- DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis? (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
