Research map / Vision-language and multimodal / Multimodal LLMs
From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation
Haowen Gu, Gensheng Pei, Junzhu Mao, Qiong Wang, Mingwu Ren, Yazhou Yao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add box guidance, SARP and R2V (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 11.35 -> 68.49 | Improved |
| add SARP and R2V on top of box guidance (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 45.02 -> 68.49 | Improved |
| add box guidance to SARP+R2V (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 36.24 -> 68.49 | Improved |
| add SARP to box guidance + R2V (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 23.22 -> 68.49 | Improved |
| add R2V to box guidance + SARP (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 45.50 -> 68.49 | Improved |
| add R2V to box guidance + SARP (ablation) | MedREAL · MedRAVS-13K overall | cIoU: 46.96 -> 70.47 | Improved |
| add only R2V without SARP vs box guidance alone (ablation) | MedREAL · MedRAVS-13K overall | gIoU: 45.02 -> 23.22 | Hurt |
| scale MLLM backbone from Qwen3-VL-2B to 4B | MedREAL · MedRAVS-13K overall | gIoU: 68.49 -> 75.96 | Improved |
| scale MLLM backbone from Qwen3-VL-2B to 4B | MedREAL · MedRAVS-13K overall | cIoU: 70.47 -> 70.26 | No clear effect |
| replace LISA with MedREAL | LISA · MedRAVS-13K overall | gIoU: 55.70 -> 68.49 | Improved |
| replace LISA with MedREAL | LISA · MedRAVS-13K BUSI | gIoU: 41.76 -> 69.15 | Improved |
| replace oracle-prompted MedSAM3 with its agent version using autonomous prompts | MedSAM3 · MedRAVS-13K overall | gIoU: 70.99 -> 55.71 | Hurt |
Closest papers by meaning
- MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.92)
- MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.91)
- Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.91)
- MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.90)
- Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine (AAAI 2025 · similarity 0.90)
- OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
