Research map / Vision-language and multimodal / Multimodal LLMs

MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the UMRG task, the U-MRG-14K dataset of implicit clinical queries with masks and CoT traces, and MedReasoner, which uses GRPO-trained Lingshu-7B to output boxes and key points that a frozen MedSAM2 converts into masks.

Problem

Medical grounding systems need explicit spatial prompts or explicit referring phrases, so they fail on implicit clinical queries; no dataset pairs such implicit queries with pixel masks and reasoning traces.

Approach

Decouple a reasoning MLLM (CRM, trained with rule-based GRPO rewards on box and key-point outputs) from a frozen promptable segmenter (ASM, MedSAM2), so the RL-trained model emits lightweight spatial prompts that a frozen segmenter converts into masks.

Key results

• MedReasoner-7B (Lingshu-7B CRM + MedSAM2) reaches 32.42 IoU, 26.55 pDice and 37.78 Dice on U-MRG-14K, versus 18.32 IoU for the best general baseline Qwen2.5-VL-72B.
• Soft-reward RL variant gives IoU 32.42 vs 9.15 for the SFT baseline on Lingshu, with zero refusals after RL.
• Among segmentation backbones, MedSAM2 gives the best Dice (37.78 with box and points) over SAM-Med2D (36.48) and MedSAM (19.00).

Surprising findings

• Adding an explicit reasoning step lowered IoU for base Qwen2.5-VL-7B and Lingshu-7B, though it cut refusals to zero for the trained model.
• Under the Soft reward, the RL-trained model yields higher IoU than the Hard reward variant, which the authors attribute to less strict alignment encouraging exploration.

Limitations

• Histology remains hard (MedReasoner IoU 11.66), so performance is uneven across super-categories.
• Dataset covers 10 modalities and 108 categories, and the authors say they are still expanding it.
• Grounding quality depends on GPT-4o-generated QA pairs, with only manual screening for consistency.

Experimental setup

Datasets: SA-Med2D-20M BiomedParse IMIS-Bench U-MRG-14K

Benchmarks: U-MRG-14K

Metrics: IoU pDice Dice refusal count

Baselines: GPT-4o Gemini-2.5-flash Qwen2.5-VL-72B InternVL3-78B MedR1-2B Lingshu-7B HuatuoGPT-7B SegZero-7B VLMR1-REC-3B SAM4MLLM-8B

Builds on: GRPO Seg-Zero MedSAM2 Lingshu

Closest papers by meaning

• CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework (ICLR 2026 · similarity 0.91)
• MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images (CVPR 2026 · similarity 0.90)
• MedSG-Bench: A Benchmark for Medical Image Sequences Grounding (NeurIPS 2025 · similarity 0.90)
• MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models (ICML 2026 · similarity 0.90)
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.90)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.89)