Research map / Large language models / Retrieval and knowledge
Expert-Guided Prompting and Retrieval-Augmented Generation for Emergency Medical Service Question Answering
Xueren Ge, Sahil Murtaza, Anthony Cortez, Homa Alemzadeh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CoT with Expert-CoT prompting (add subject area and certification level) | OpenBioLLM-70B · EMSQA Public test | accuracy: 59.88 -> 61.92 | Improved |
| replace CoT with Expert-CoT prompting | Qwen3-32B · EMSQA Public test | accuracy: 84.96 -> 85.70 | Improved |
| replace CoT with Expert-CoT prompting | Llama-3.3-70B · EMSQA Public test | accuracy: 81.89 -> 82.42 | Improved |
| use Filter-predicted instead of ground-truth expertise in Expert-CoT (ablation) | Qwen3-32B · EMSQA Public test | accuracy: 85.70 -> 85.57 | No clear effect |
| add subject-area filtered retrieval (ExpertRAG FTR/RTF) instead of global retrieval (ablation) | Qwen3-4B RAG with CoT · EMSQA Public test | accuracy: 78.12 -> 81.11 | Improved |
| add Expert-CoT to global RAG (ablation) | Qwen3-4B global RAG · EMSQA Public test | accuracy: 78.12 -> 79.59 | Improved |
| combine ExpertRAG (RTF) with Expert-CoT using ground-truth expertise (ablation) | Qwen3-4B · EMSQA Public test | accuracy: 78.12 -> 82.24 | Improved |
| use Filter-predicted instead of ground-truth expertise in ExpertRAG (ablation) | Qwen3-4B ExpertRAG RTF + Expert-CoT · EMSQA Public test | accuracy: 82.24 -> 80.95 | Hurt |
| add KB retrieval instead of patient-record retrieval (ablation) | Qwen3-4B RAG · EMSQA Public test | accuracy: 73.02 -> 76.49 | Improved |
| combine KB and patient records in global retrieval (ablation) | Qwen3-4B RAG · EMSQA Public test | accuracy: 76.49 -> 78.12 | Improved |
| increase few-shot examples from 4 to 32 | Qwen3-32B · EMSQA Public test | accuracy: 84.41 -> 81.13 | Hurt |
| train Filter with LoRA and two classification heads instead of prompting an LLM for expertise | Qwen3-4B · EMSQA Public subject-area classification | micro F1: 59.72 -> 80.72 | Improved |
Closest papers by meaning
- MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning (NeurIPS 2024 · similarity 0.88)
- MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records (AAAI 2024 · similarity 0.87)
- ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room (NeurIPS 2026 · similarity 0.87)
- Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Search (NeurIPS 2025 · similarity 0.86)
- MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey (NeurIPS 2024 · similarity 0.86)
- EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
