Research map / Vision-language and multimodal / Multimodal LLMs

MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds MedMO, an open medical MLLM from Qwen3-VL-8B via four post-training stages (medical SFT, high-res grounding SFT, instruction tuning, GRPO/DAPO with a box-level GIoU-based reward), reporting SOTA-level VQA, QA, report generation and grounding among open medical models.

Problem

Medical MLLMs lack broad domain/modality coverage, reliable grounded reasoning, and spatial localization; existing models rely on distilled proprietary data and narrow task scope.

Approach

Multi-stage post-training of Qwen3-VL on a 26M-sample, 45-dataset medical corpus, with an RL stage using a verifiable Hungarian-matched bounding-box reward (L1 + GIoU) plus label accuracy for spatial grounding.

Key results

• MedMO-8B-Next reaches a VQA average of 72.7% vs. 66.1% for Fleming-VL-8B and a QA average of 60.1% vs. 45.7%.
• MedMO-8B-Next achieves 56.1 IoU on Bacteria grounding vs. 9.1 for Fleming-VL-8B (per the figure caption) and a grounding average of 56.8% across six tasks vs. 17.2% for Fleming-VL-8B.
• On MIMIC-CXR, MedMO-8B-Next reaches CIDEr 143.4 vs. 132.5 for Fleming-VL-8B; on Med-Trinity its CIDEr is 272.1 vs. 81.5 for the next best open model.

Surprising findings

• Fleming-VL-8B scores 0.00 IoU on NIH and DeepLesion grounding, while MedMO-8B reaches 38.5 on DeepLesion.
• Stage-wise training causes minor task-level degradation (catastrophic forgetting), and the RL stage with DAPO produced only small gains on some grounding sets (e.g., +0.4 on Bacteria and DeepLesion).

Limitations

• Stage-wise training causes minor task-level performance shifts from catastrophic forgetting.
• Grounding performance on NIH chest X-ray remains weak (MedMO-8B-Next 15.9 IoU vs. Qwen3VL-8B 16.4).
• Some reported numbers are inconsistent across text and captions (e.g., Bacteria IoU gain stated as +47.0 and +47.8), and the paper lacks clear release statements.

Experimental setup

Datasets: MedTrinity IU-Xray MIMIC-CXR CheXpert Plus PMC-VQA PathVQA SLAKE VQA-RAD MedQA PubMedQA MedMCQA NIH Chest X-ray

Benchmarks: MMMU-Med VQA-RAD SLAKE PathVQA PMC-VQA OmniMedVQA MedXpertQA MMLU-Med PubMedQA MedMCQA MedQA Medbullets

Metrics: accuracy ROUGE-L CIDEr RaTE Semb IoU

Baselines: Fleming-VL-8B Lingshu-7B MedGemma-4B-IT HuatuoGPT-V-7B Qwen3-VL-8B InternVL3-8B GPT-4.1 Claude Sonnet 4 Gemini-2.5-Flash MedSG-Bench

Builds on: Qwen3-VL GRPO DAPO MedTrinity GIoU

Closest papers by meaning

• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.91)
• Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks (ICLR 2025 · similarity 0.91)
• MIMO: A Medical Vision Language Model with Visual Referring Multimodal Input and Pixel Grounding Multimodal Output (CVPR 2025 · similarity 0.90)
• MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models (ICLR 2025 · similarity 0.90)
• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.90)
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.90)