Research map / Large language models / Retrieval and knowledge
Towards Error-Free EHRs: Reasoning-Intensive Consistency Verification Between Clinical Notes and Structured Tables in Electronic Health Records
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EHR-ReasonCon, a reasoning-intensive note-table consistency benchmark on MIMIC-III (8,048 entities), and EHR-Inspector, a tool-augmented LLM framework that outperforms CheckEHR across four backbones.
Problem
Existing note-table consistency verification in EHRs relies on surface-level matching of numeric values or simple events, failing to capture clinical interpretation, event relations, and temporal changes.
Approach
A reasoning-intensive benchmark with expert-guided annotation supported by table-exploration tools, plus an LLM framework that mirrors the annotation workflow: note segmentation, temporal reference extraction, multi-stage entity extraction with ontology guidance, scope filtering, and tool-augmented verification with a validation cache.
Key results
- On EHR-ReasonCon with Gemini 2.5 Flash, EHR-Inspector reaches 77.36 lenient recall and 71.67 lenient precision on discharge notes, versus 48.80 / 65.14 for CheckEHR.
- Ablation with Gemini 2.5 Flash: removing segmentation drops lenient F1 from 74.75 to 69.75; removing scope filtering drops it to 70.04.
- On EHRCon, EHR-Inspector achieves lenient F1 of 86.41, outperforming CheckEHR by 24.99 points.
Surprising findings
- Removing the entity-to-table item alignment tools slightly improved lenient performance (F1 76.00 vs 74.75), attributed to LLMs not needing space-reduction strategies that humans use.
- MedGemma 27B, a medical-specialized model, does not outperform general-purpose reasoning models and underperforms under CheckEHR.
Limitations
- Limited number of clinical notes (105) may constrain representativeness.
- Reliance on preprocessed MIMIC data may not capture real-world clinical error complexity.
- LLM-as-a-judge paradigm may introduce model-driven biases.
Experimental setup
Datasets: MIMIC-III EHRCon MIMIC-IV discharge summaries
Benchmarks: EHR-ReasonCon EHRCon MIMIC-IV discharge summaries
Metrics: recall precision F1
Baselines: CheckEHR Trained NER BERT Ensemble
Builds on: EHRCon CheckEHR
Closest papers by meaning
- EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records (NeurIPS 2024 · similarity 0.91)
- ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room (NeurIPS 2026 · similarity 0.84)
- Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.84)
- One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models (NeurIPS 2026 · similarity 0.83)
- EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summaries (NeurIPS 2024 · similarity 0.82)
- From Conversation to Query Execution: Benchmarking User and Tool Interactions for EHR Database Agents (ICLR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.