Research map / Vision-language and multimodal / Multimodal LLMs

MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MedVR, an annotation-free agentic RL framework that lets medical VLMs interleave text reasoning with zoom-in tool calls, using entropy-guided exploration (EVR) and consensus-based reward (CCA).

Problem

Medical VLMs trained with text-only RLVR fail to ground reasoning in visual evidence and are prone to visual hallucination, while adding visual tool use typically requires costly intermediate grounding annotations.

Approach

Trains a VLM with GRPO to call a Zoom-in tool during reasoning, branching rollouts at high token-entropy tool-call moments (EVR) and rewarding trajectories whose zoom boxes agree with the majority consensus mask of successful rollouts (CCA), removing the need for grounding labels.

Key results

• MedVR (Qwen2.5-VL-7B) reaches 96.8 on OmniMedVQA and 59.2 average on multiple-choice benchmarks, the best reported in the main table.
• On free-text tasks MedVR averages 74.0 vs 70.3 for Lingshu-7B, including 62.3 on out-of-domain PathVQA.
• Ablation: full MedVR gives 96.77 OmniMedVQA / 54.31 PMC-VQA / 26.38 MedXpertQA vs 94.50 / 53.40 / 21.38 for textual RL.

Surprising findings

• Adding the Zoom-in tool without EVR or CCA slightly lowers OmniMedVQA and PMC-VQA performance relative to textual RL.
• Localization-trained MedVR is nearly as accurate as an oracle variant using ground-truth masks on GEMEX-ThinkVG (79.08 vs 79.62 accuracy).

Limitations

• Only one visual operation (Zoom-in) is studied.
• Evaluation is on a limited set of medical VQA benchmarks, and the OOD claims are contested by the paper's own appendix on baseline data overlap.
• Consensus pseudo-labels rely on majority agreement among successful rollouts, which may propagate shared biases.

Experimental setup

Datasets: OmniMedVQA VQA-RAD SLAKE GEMEX-ThinkVG RefCOCO Geometry3k

Benchmarks: OmniMedVQA PMC-VQA MedXpertQA VQA-RAD SLAKE PathVQA GEMEX-ThinkVG ChestX-ray8 ISIC refCOCO MathVista MathVerse

Metrics: accuracy F1 / free-text accuracy mIoU

Baselines: Qwen2.5-VL-7B Qwen2.5-VL-32B InternVL2.5-8B InternVL3-8B InternVL3-14B Med-R1-2B MedVLM-R1-2B MedGemma-4B LLaVA-Med-7B HuatuoGPT-V-7B Lingshu-7B DeepEyes

Builds on: GRPO DeepEyes Med-R1 Qwen2.5-VL

Closest papers by meaning

• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.89)
• CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework (ICLR 2026 · similarity 0.89)
• MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning (ICLR 2026 · similarity 0.89)
• MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis (AAAI 2026 · similarity 0.89)
• Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning (ICML 2026 · similarity 0.88)
• QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training (NeurIPS 2025 · similarity 0.86)