Research map / Vision-language and multimodal / Multimodal LLMs

Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Med-Scout applies GRPO post-training with three unlabeled-image geometric proxy tasks (scale localization, 2x2 jigsaw, anomaly detection) and a dense geometric reward, plus a new Med-Scout-Bench, to cure MLLM "geometric blindness" in medical images.

Problem

General and medical MLLMs fail to ground outputs in geometric constraints of medical images (scale consistency, topology, structural anomalies), producing fluent but factually wrong descriptions.

Approach

Derive verifiable RL supervision from unlabeled medical images via three geometric proxy tasks inspired by clinician reading patterns, and use a dense, continuous geometric reward (IoU, element-wise match, distance-based exp reward) within GRPO instead of sparse binary rewards.

Key results

• On Med-Scout-Bench, Qwen3-VL-8B-Instruct average rises from 39.7 to 83.6 after Med-Scout, while GPT-5 scores 63.6 and Gemini-3-Flash 66.1.
• Lingshu-7B improves from 31.9 to 71.9 average on Med-Scout-Bench according to the text (table not fully shown).
• On radiological and generalization VQA, gains are typically 1-4 points (e.g., Qwen3-VL-4B Rad-VQA 41.5 to 45.7; Lingshu-7B VQA-RAD 68.9 to 71.0).

Surprising findings

• Direct mode and reasoning (CoT) mode perform nearly identically, suggesting explicit CoT is not strictly necessary.
• Qwen3-VL-4B reaches higher Med-Scout-Bench average (84.4) than the 8B counterpart (83.6).

Limitations

• Med-Scout-Bench is built from the same synthetic proxy-task generators as training, so in-distribution gains may overstate clinical capability.
• Reasoning-mode reward does not check logical validity, so the CoT benefit is unclear.
• Evaluation relies on LLM-as-a-judge (Gemini-3-Flash) and the pilot study uses only two models with 200 samples each.

Experimental setup

Datasets: TotalSegmentator MIMIC-CXR RadImageNet-VQA VQA-RAD SLAKE IU-Xray PMC-VQA OmniMedVQA MedXpertQA

Benchmarks: Med-Scout-Bench RadImageNet-VQA VQA-RAD SLAKE MIMIC-CXR IU-Xray PMC-VQA OmniMedVQA MedXpertQA

Metrics: accuracy DGR score ROUGE-L CIDEr SemScore

Baselines: GPT-5 Gemini-3-Flash InternVL3-8B Qwen2.5-VL-7B-Instruct Qwen3-VL-8B-Instruct Lingshu-7B HuatuoGPT-Vision-7B MedGemma-4B-IT LLaVA-Med-7B Jigsaw-R1 ViCrit

Builds on: GRPO Jigsaw-R1 Visual Jigsaw ViCrit BiomedCLIP

Closest papers by meaning

• MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models (ICML 2026 · similarity 0.88)
• 3DMedAgent: Unified Perception-to-Understanding for 3D Medical Analysis (ICML 2026 · similarity 0.87)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.86)
• Anatomical Region-Guided Contrastive Decoding: A Plug-and-Play Strategy for Mitigating Hallucinations in Medical VLMs (AAAI 2026 · similarity 0.86)
• Enhancing Geometric Perception in VLMs via Translator-Guided Reinforcement Learning (ICLR 2026 · similarity 0.86)
• GePBench: Evaluating Fundamental Geometric Perception for Multimodal Large Language Models (ICML 2026 · similarity 0.86)