Research map / Vision-language and multimodal / Multimodal LLMs

MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 13 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MedAgent-Pro, a hierarchical agentic workflow that generates RAG-grounded disease-level diagnostic plans and executes them at patient level with visual tools and step-wise reliability checks for evidence-based multi-modal medical diagnosis.

Problem

Existing VLM and medical agent systems answer medical diagnosis as one-hop VQA with empirical judgments, lacking guideline alignment, quantitative analysis, and verified step-by-step reasoning.

Approach

Decouples diagnosis into disease-level guideline-grounded planning (RAG over MedlinePlus, executable JSON plan of tool calls) and patient-level execution where specialized tools compute clinical indicators, each step is checked for reliability, and a risk-weighted score yields the diagnosis.

Key results

• On glaucoma diagnosis (REFUGE2) MedAgent-Pro reaches 90.4% bAcc and 76.4% F1, versus 56.4% and 21.1% for GPT-4o.
• On NEJM, MedAgent-Pro achieves 81.7% accuracy vs 70.9% for GPT-4o and 73.8% for the best medical agent baseline (MDAgent).
• On MIMIC chest X-ray 12-task average, MedAgent-Pro gets 72.0% bAcc vs 58.3% for GPT-4o.

Surprising findings

• Replacing GPT-4o with the ophthalmology-specific VisionUnite for qualitative analysis gives only marginal gains (bAcc 92.9 vs 90.4), suggesting general VLMs suffice when guided by guidelines.

Limitations

• Depends on availability of visual tools, which are limited in some medical domains.
• Qualitative analysis relies on VLMs prone to inconsistency and hallucination.
• Non-clinical images lack compatible tool support.

Experimental setup

Datasets: REFUGE2 MITEA MIMIC-CXR NEJM MedlinePlus

Benchmarks: REFUGE2 MITEA MIMIC NEJM

Metrics: balanced accuracy F1 accuracy AUC

Baselines: GPT-4o Janus-Pro-7B LLaVA-Med BioMedCLIP Qwen2.5-VL-7B InternVL2.5-8B MedAgents MMedAgent MDAgent RetiZero VisionUnite CheXagent

Builds on: MedAgents MMedAgent MDAgent Medical SAM Adapter MedSAM

Closest papers by meaning

• CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework (ICLR 2026 · similarity 0.89)
• MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making (NeurIPS 2024 · similarity 0.89)
• MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks (NeurIPS 2025 · similarity 0.89)
• RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis (NeurIPS 2025 · similarity 0.88)
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.88)
• MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments (NeurIPS 2026 · similarity 0.88)