Research map / Large language models / Agents, code and math
PatientSim: A Persona-Driven Simulator for Realistic Doctor-Patient Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PatientSim, an LLM-based patient simulator grounded in MIMIC-IV/ED/Note profiles with 37 persona combinations across four axes, evaluated for persona fidelity and factuality across eight LLMs with clinician validation.
Problem
Existing LLM patient simulators fail to reflect the diverse, clinically relevant patient personas seen in ED consultations and rarely evaluate persona fidelity or factual accuracy, which limits their use for training or evaluating doctor LLMs.
Approach
A patient simulator whose behavior is driven by real MIMIC-derived clinical profiles plus a 37-persona space defined by personality, CEFR-based language proficiency, medical history recall, and cognitive confusion, with sentence-level NLI factuality and LLM-judged plausibility of unsupported statements.
Key results
- Llama 3.3 70B had the highest average persona fidelity among open models (3.68 avg) and top-tier sentence-level entailment (98.1%) with lowest contradiction (1.9%).
- Four clinicians rated the Llama 3.3 70B simulator at an average of 3.89 out of 4 across six criteria.
- Clinicians rated plausibility of unsupported sentences at 3.91 out of 4 on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Llama-3.3-70B-Instruct rather than Qwen2.5-7B-Instruct as the simulator backbone | PatientSim · Unsupported statements | Plausibility: 3.862 -> 3.963 | Helped |
| Use a larger LLM backbone (70B-class rather than 8B-class) | PatientSim · Supported statements | Entail (%): 0.939 -> 0.981 | Helped |
| Use a larger LLM backbone (70B-class rather than 8B-class) | PatientSim · Unsupported statements | Plausibility: 3.862 -> 3.963 | Helped |
| Select Llama 3.3 70B as the simulator backbone | PatientSim · Persona fidelity | Avg.: 3.34 -> 3.68 | Helped |
Surprising findings
- General LLM benchmark performance does not always correlate with simulation fidelity; Llama 3.1 8B showed better persona fidelity than Qwen2.5 72B.
- Most models struggled to portray negative emotions such as impatience and distrust.
Limitations
- Based solely on MIMIC database, limiting generalizability.
- Text-only simulation cannot capture non-verbal expressions.
- Human evaluation involved only four clinicians.
Experimental setup
Datasets: MIMIC-IV MIMIC-IV-ED MIMIC-IV-Note
Benchmarks: PatientSim persona evaluation (37 personas) PatientSim factuality evaluation (170 MIMIC-derived profiles)
Metrics: persona fidelity (4-point) Entail % Contradict % Information Coverage Information Consistency plausibility (4-point) Gwet's AC1
Baselines: Gemini-2.5-Flash GPT-4o mini DeepSeek-R1-Distill-Llama-70B Qwen2.5-72B-Instruct Llama-3.1-70B-Instruct Llama-3.1-8B-Instruct Qwen2.5-7B-Instruct
Builds on: MIMIC-IV MIMIC-IV-ED MIMIC-IV-Note CEFR
Closest papers by meaning
- PatientVLM Meets DocVLM: Pre-Consultation Dialogue Between Vision-Language Models for Efficient Diagnosis (AAAI 2026 · similarity 0.84)
- Doctor-R1: Mastering Clinical Inquiry with Experiential Agentic Reinforcement Learning (ICLR 2026 · similarity 0.84)
- CareBot: A Pioneering Full-Process Open-Source Medical Language Model (AAAI 2025 · similarity 0.83)
- MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey (NeurIPS 2024 · similarity 0.82)
- Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue (AAAI 2024 · similarity 0.82)
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.