Research map / Multi-agent systems and games / LLM agents

Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Doctor-R1 trains an 8B doctor agent (Qwen3-8B) with GRPO in a multi-agent simulated consultation environment, using a two-tiered process/outcome reward and an experience repository of high-reward trajectories, and reports gains over larger medical LLMs and proprietary models on HealthBench and MAQuE.

Problem

Medical LLMs perform well on static medical QA but lack strategic, multi-turn, empathetic patient inquiry and differential diagnosis needed for dynamic clinical consultation.

Approach

Combines multi-turn agentic RL (GRPO) with a veto-based multi-dimensional turn-level process reward for soft skills, an outcome reward for diagnostic correctness, and a multi-stage experience retrieval repository (similarity + reward + novelty filtering) that supplies high-reward past trajectories to the doctor agent.

Key results

• Doctor-R1 (8B) scores 36.29 on HealthBench Main vs. 33.16 for Baichuan-M2-32B and 31.18 for GPT-4.1.
• On MAQuE, Doctor-R1 reaches 60.00 accuracy and 93.80 empathy, versus 57.00 and 65.20 for Baichuan-M2-32B.
• Doctor-R1 reaches 83.50 on MedQA vs. 63.50 for its Qwen3-8B base model.

Surprising findings

• Enhancing inquiry/communication capability appears to improve diagnostic accuracy, with ablations showing removing process reward lowers accuracy.
• Similarity-only experience retrieval gives only marginal gains over no experience.

Limitations

• Process reward and outcome judging rely on an LLM judge (Qwen3-8B), serving only as proxy for human expertise.
• Performance on HealthBench Hard remains well below GPT-5 and Baichuan-M2-32B.
• Patient agent and evaluator are LLM-based, so simulated dialogues may not reflect real patients.

Experimental setup

Datasets: KaMed HealthBench MAQuE MedQA MMLU

Benchmarks: HealthBench MAQuE MedQA MMLU

Metrics: average score accuracy communication quality context awareness task success accuracy empathy MedQA accuracy MMLU accuracy

Baselines: GPT-4.1 GPT-5 Claude Sonnet 4 Gemini-2.5-Flash Grok-4 Baichuan-M2-32B UltraMedical-70B HuatuoGPT-o1-70B DoctorAgent-RL HuatuoGPT-o1-8B Med42-v2-8B UltraMedical-8B

Builds on: GRPO PPO VeRL DoctorAgent-RL

Closest papers by meaning

• MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes (NeurIPS 2026 · similarity 0.88)
• Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue (AAAI 2024 · similarity 0.88)
• MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning (ICLR 2026 · similarity 0.88)
• MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments (NeurIPS 2026 · similarity 0.88)
• Why Specialist Models Still Matter: A Heterogeneous Multi-Agent Paradigm for Medical Artificial Intelligence (ICML 2026 · similarity 0.87)
• MedChain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence (NeurIPS 2025 · similarity 0.87)