Research map / Vision-language and multimodal / Multimodal LLMs

CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CARE, a three-stage evidence-grounded agentic framework for medical VQA (entity proposal VLM, entity-referring segmentation, evidence-grounded VQA) with a GPT-5 coordinator for tool planning and CoT review, reaching 77.5% average accuracy.

Problem

Medical VLMs act as end-to-end black boxes that skip explicit localization of visual evidence, leading to shortcut learning, hallucination, and limited clinical accountability in medical visual question answering.

Approach

Decompose medical VQA into coordinated specialists: an RLVR-trained entity proposal VLM with embedding-similarity reward and Kuhn-Munkres matching, a SAM-based referring segmenter producing ROI masks with confidence, and an EG-VQA VLM that consumes zoom-in, mask, or global evidence views; a VLM coordinator plans tool calls and reviews CoT-answer consistency.

Key results

• CARE-Flow-B (10B) reaches 74.91% average accuracy, versus 66.66% for Lingshu-7B and 72.29% for Lingshu-32B.
• CARE-Coord-B with GPT-5 coordinator reaches 77.54% average accuracy, 5.2 points above Lingshu-32B.
• Ablation: SFT+DAPO+length reward raises EG-VQA overall from 65.3% to 74.9%; the coordinator with review gives 77.5% versus 74.9% for majority-vote CARE-Flow.

Surprising findings

• Using a stronger proprietary coordinator with CoT review can overwrite correct expert answers in up to 3.05% of cases, though net gains remain positive (+1.79%).
• Greedy matching or binary rewards for entity proposal performed worse than KM matching with similarity reward, and GPT-5 direct entity proposal was suboptimal.

Limitations

• Coordinator relies on proprietary GPT-5 with high latency (about 43.5 s per query) and hallucination risk.
• Underperforms specialized models on VQA-RAD open-ended recall and on SLAKE relative to some baselines; method suited to local-detail tasks rather than global ones.
• Evaluation of open-ended answers uses GPT-4o as judge; coordinator review gains partly depend on GPT-5 reasoning.

Experimental setup

Datasets: SA-Med-20M OmniMedVQA VQA-RAD SLAKE MeCo-G VQA-Med-2019

Benchmarks: OMVQA-3k VQA-RAD SLAKE VQA-Med-2019 MeCo-G

Metrics: accuracy mean Dice score entity accuracy recall

Baselines: GPT-4o GPT-5 Lingshu-32B Lingshu-7B HuatuoGPT-Vision-34B MedGemma-27B InternVL3-38B DeepEyes-7B MedVLM-R1-2B BiomedParse

Builds on: SA-Med-2D DAPO GRPO InternVL3 BiomedParse

Closest papers by meaning

• MedReasoner: Reinforcement Learning Drives Reasoning Grounding from Clinical Thought to Pixel-Level Precision (AAAI 2026 · similarity 0.91)
• Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning (CVPR 2026 · similarity 0.90)
• MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow (ICLR 2026 · similarity 0.89)
• Medical thinking with multiple images (ICLR 2026 · similarity 0.89)
• OpenMedReason: Scientific Reasoning Supervision for Medical Vision–Language Models (NeurIPS 2026 · similarity 0.89)
• Visual and Domain Knowledge for Professional-level Graph-of-Thought Medical Reasoning (ICML 2025 · similarity 0.89)