Research map / Multi-agent systems and games / LLM agents

RareAgents: Autonomous Multi-disciplinary Team for Rare Disease Diagnosis and Treatment

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RareAgents, an LLM multi-agent framework (MDT collaboration, dynamic long-term memory, medical tools) on Llama-3.1-8B/70B for rare disease differential diagnosis and medication recommendation, plus a new MIMIC-IV-Ext-Rare dataset.

Problem

Diagnosing and treating rare diseases in realistic open-ended clinical settings, where existing medical LLM agents are mostly evaluated on multiple-choice QA and lack memory, tool use, and multi-specialist collaboration.

Approach

An attending physician agent assembles a multi-disciplinary team from a predefined pool of 41 specialist agents; each agent has dynamic long-term memory (retrieval of similar cases or prior visits) and access to diagnostic/drug tools (Phenomizer, LIRICAL, PhenoBrain, DrugBank, DDI-graph), and the team reaches consensus over iterative rounds before a final decision.

Key results

• RareAgents (Llama-3.1-70B) reaches Hit@1 0.5589 and MR 1.0 on RareBench-Public diagnosis, versus 0.4042 for the best medical multi-agent baseline (MDAgents, 70B).
• On MIMIC-IV-Ext-Rare medication recommendation, RareAgents (70B) achieves Jaccard 0.4108 and F1 0.5563, above the best trained baseline (G-Bert, Jaccard 0.4030) and GPT-4o (Jaccard 0.3282).
• Ablation with Llama-3.1-70B shows removing memory drops diagnosis Hit@1 from 0.5589 to 0.4336, the largest drop among the three modules.

Surprising findings

• Removing the tools module raised DDI rate on treatment by about 23% for the 8B model, while OpenBioLLM-8B had the lowest DDI but poor Jaccard and F1.
• Domain-specific fine-tuned medical LLMs such as OpenBioLLM performed worse than the base Llama-3.1 on the diagnosis task, suggesting fine-tuning does not generalize across medical tasks.

Limitations

• Uses only pre-extracted symptom and code text; no imaging or genotype data.
• No domain-specific fine-tuning of the underlying LLMs; purely prompt and agent based.
• MIMIC-IV-Ext-Rare is pending PhysioNet review and not yet publicly available; evaluation relies on GPT-4o as an external judge for diagnosis matching.

Experimental setup

Datasets: RareBench-Public MIMIC-IV MIMIC-IV-Ext-Rare

Benchmarks: RareBench-Public MIMIC-IV-Ext-Rare

Metrics: Hit@1 Hit@3 Hit@10 MR Jaccard F1 DDI rate #MED

Baselines: GPT-4o GPT-3.5 UltraMedical OpenBioLLM DeepSeek-R1-Distill-Llama Baichuan-M1-14B HuatuoGPT-o1-70B MedAgents MDAgents Phenomizer LIRICAL Phen2Disease

Builds on: MedAgents MDAgents Agent Hospital RAREMed chain-of-thought

Closest papers by meaning

• MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making (NeurIPS 2024 · similarity 0.87)
• MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow (ICLR 2026 · similarity 0.85)
• MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks (NeurIPS 2025 · similarity 0.84)
• Why Specialist Models Still Matter: A Heterogeneous Multi-Agent Paradigm for Medical Artificial Intelligence (ICML 2026 · similarity 0.84)
• MTBBench: A Multimodal Sequential Clinical Decision-Making Benchmark in Oncology (NeurIPS 2025 · similarity 0.83)
• MedChain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence (NeurIPS 2025 · similarity 0.83)