Research map / Large language models / AI and society
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MIMIC-ED-Assist, a benchmark for cost-effective lab test recommendation in the ED, and ED-Copilot, a BioGPT-based model fine-tuned with SFT and then PPO to pick lab groups that balance prediction accuracy against wait time.
Problem
Emergency department crowding is driven by lengthy laboratory testing before diagnosis; AI systems need to recommend informative lab groups that minimize time-cost while accurately predicting critical outcomes and lengthened ED stay.
Approach
Linearize triage and lab results into text, fine-tune a biomedical pre-trained LM (BioGPT) to predict next lab group and outcomes, then apply PPO over the LM's [EOS] representations to trade off F1 against cumulative lab time-cost, enabling personalized sequential test recommendation.
Key results
- ED-Copilot achieves F1 0.413 and AUC 0.820 for critical outcome, versus 0.394 F1 for the best tree baseline (LightGBM), while halving average lab time-cost from 265 to 125 minutes.
- For lengthened ED stay, ED-Copilot reaches F1 0.232 and AUC 0.707 with 154 minutes average time-cost, compared with 265 minutes for tree baselines.
- Replacing BioGPT with Llama-7B (LoRA) without RL gives comparable accuracy to BioGPT without RL but no time-cost reduction (265 minutes).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ED-Copilot instead of the strongest tree baseline | MIMIC-ED-Assist · Critical Outcome | F1: 0.394 (0.016) -> 0.413 (0.028) | Helped |
| use ED-Copilot instead of the strongest tree baseline | MIMIC-ED-Assist · Lengthened ED Stay | F1: 0.217 (0.015) -> 0.232 (0.023) | Helped |
| use ED-Copilot instead of LightGBM | MIMIC-ED-Assist · Critical Outcome | Avg. Time-cost: 265 Min -> 125 (21) Min | Helped |
| use ED-Copilot instead of LightGBM | MIMIC-ED-Assist · Lengthened ED Stay | Avg. Time-cost: 265 Min -> 154 (33) Min | Helped |
| use standard feature names rather than raw laboratory test names (ablation) | ED-Copilot · Critical Outcome | F1: 0.397 -> 0.413 | Helped |
| add triage features (ablation) | ED-Copilot · Critical Outcome | F1: 0.277 -> 0.413 | Helped |
| add triage features (ablation) | ED-Copilot · Lengthened ED Stay | F1: 0.145 -> 0.252 | Helped |
| add CBC features (ablation) | ED-Copilot · Critical Outcome | F1: 0.385 -> 0.413 | Helped |
| add CBC features (ablation) | ED-Copilot · Lengthened ED Stay | F1: 0.224 -> 0.252 | Helped |
| add CHEM features (ablation) | ED-Copilot · Critical Outcome | F1: 0.420 vs 0.413 | No clear effect |
| add CHEM features (ablation) | ED-Copilot · Lengthened ED Stay | F1: 0.234 -> 0.252 | Helped |
| add reinforcement learning (ablation) | BioGPT (345M) · Critical Outcome | F1: 0.381 -> 0.413 | Helped |
Surprising findings
- Llama-7B (LoRA) without RL performs comparably to 345M BioGPT without RL, yet it lacks any time-cost reduction, showing that scale alone does not address testing cost.
- Unrestricted lab group selection underperforms the restricted version at small time budgets but matches it as the allowed time grows.
Limitations
- MIMIC-ED-Assist is retrospective and only includes hospitalized ED patients, so it is biased toward severe cases.
- Time-cost is approximated by summing average lab group completion times, which ignores parallel testing and true ED LOS dynamics.
- Data from a single center (Beth Israel Deaconess Medical Center) may not generalize to other health systems.
Experimental setup
Datasets: MIMIC-IV MIMIC-IV-ED
Benchmarks: MIMIC-ED-Assist
Metrics: F1 AUC sensitivity specificity average time-cost (minutes)
Baselines: Random Forest XGBoost LightGBM 3-layer DNN SM-DDPO
Builds on: BioGPT PPO SM-DDPO MIMIC-IV
Closest papers by meaning
- ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room (NeurIPS 2026 · similarity 0.84)
- Timely Clinical Diagnosis through Active Test Selection (NeurIPS 2025 · similarity 0.83)
- Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models (NeurIPS 2025 · similarity 0.82)
- MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments (NeurIPS 2026 · similarity 0.82)
- Instruction Tuning Large Language Models to Understand Electronic Health Records (NeurIPS 2024 · similarity 0.81)
- Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML (ICLR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.