Research map / Vision-language and multimodal / Multimodal LLMs

MedEyes: Learning Dynamic Visual Focus for Medical Progressive Diagnosis

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MedEyes, an RL framework for medical VQA that mixes off-policy expert visual-search trajectories (scanning/drilling navigator, nucleus-sampled trajectories) with on-policy rollouts under a dual-stream GRPO that normalizes advantages separately per source.

Problem

Medical VLMs trained with SFT overfit to memorized trajectories, and on-policy RLVR for visual chain-of-thought reinforces clinically inaccurate reasoning paths and suffers from advantage collapse and entropy collapse.

Approach

Converts expert region-level visual search traces into off-policy multi-round grounding trajectories (scanning and drilling modes, nucleus sampling with adaptive termination) and trains with GRPO that computes advantage statistics separately for on-policy and off-policy trajectories.

Key results

• MedEyes reaches 65.9 average accuracy across five medical VQA benchmarks, reported as +8.5 over GMAI-VL and +13.4 over MedVLM-R1.
• Ablation: removing off-policy trajectories drops the three-benchmark average from 71.5 to 61.0.
• MedEyes gets 70.7 on VQA-RAD, 79.1 on SLAKE, and 64.8 on PathVQA.

Surprising findings

• The reported average gaps are inconsistent: the table shows 65.9 vs 57.4 for GMAI-VL (about 8.5 points) but the 'Delta' column is given as relative percentages of the same values, and the ablation averages use only three benchmarks while the main table averages five.

Limitations

• Fails on quantitative measurement tasks such as tumor sizing due to flawed scale estimation.
• Confuses fine-grained pathological concepts (e.g., aneurysmal changes vs dissecting aneurysms).
• Relies on a single visual expert (MedPLIB) for generating trajectories and the paper gives limited detail on the training data.

Experimental setup

Datasets: VQA-RAD SLAKE PathVQA PMC-VQA MMMU (Health & Medicine)

Benchmarks: VQA-RAD SLAKE PathVQA PMC-VQA MMMU (Health & Medicine)

Metrics: accuracy

Baselines: GPT-4o Qwen2.5-VL-3B InternVL-2 RadFM MedVInT LLaVA-Med Med-Flamingo GMAI-VL GRIT DeepEyes Med-R1 MedVLM-R1

Builds on: GRPO DeepEyes GRIT Med-R1

Closest papers by meaning

• MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning (ICLR 2026 · similarity 0.89)
• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.89)
• Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning (ICML 2026 · similarity 0.87)
• MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow (ICLR 2026 · similarity 0.87)
• QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training (NeurIPS 2025 · similarity 0.87)
• MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning (ICLR 2026 · similarity 0.86)