Research map / Large language models / Reasoning and chain of thought

Toward Better EHR Reasoning in LLMs: Reinforcement Learning with Expert Attention Guidance

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EAG-RL, a two-stage method that improves LLM EHR reasoning by SFT on expert-guided MCTS reasoning trajectories followed by RL with a Jaccard attention-alignment reward from a Concare expert model and entropy-aware adaptive upper clipping.

Problem

LLMs underperform on EHR-based clinical prediction because they struggle with temporally structured, high-dimensional tabular data, and existing hybrid approaches use LLMs only as frozen priors rather than improving their intrinsic reasoning.

Approach

Distills stepwise clinical reasoning trajectories via expert-guided MCTS to initialize the policy, then runs GRPO-style RL with a reward that aligns the LLM's extracted salient features with those highlighted by an expert Transformer EHR model (Concare), plus an entropy-dependent adaptive upper clip bound (DAPO-style).

Key results

• EAG-RL (Qwen2.5-7B) reaches 87.70 AUROC / 82.95 AUPRC on TJH mortality, versus 71.99 / 64.27 for vanilla prompting and 81.20 / 75.14 for SFT.
• On MIMIC-IV mortality with Qwen2.5-3B, EAG-RL reaches 70.40 AUROC versus 51.10 for vanilla prompting.
• Authors report an average 14.62% improvement over prompt-based approaches across models and tasks.

Surprising findings

• Removing the RL stage with Stage-1 only (w/o Stage-2) causes a larger drop than removing Stage-1, per the ablation.
• Think-then-Answer prompting underperforms vanilla prompting on several LLaMA3.1-8B and Qwen2.5-3B settings.

Limitations

• Evaluated on only two EHR datasets and tasks.
• Models limited to up to 8B parameters.
• Relies on a single expert EHR model (Concare); no multi-expert supervision.

Experimental setup

Datasets: MIMIC-IV TJH

Benchmarks: TJH mortality MIMIC-IV mortality MIMIC-IV readmission

Metrics: AUROC AUPRC

Baselines: Vanilla prompting Think & Answer SFT GRPO DAPO HuatuoGPT-o1-7B OpenBioLLM-8B DeepSeek-R1-7B Concare

Builds on: GRPO DAPO Monte Carlo Tree Search Concare Think-then-Answer

Closest papers by meaning

• Instruction Tuning Large Language Models to Understand Electronic Health Records (NeurIPS 2024 · similarity 0.89)
• MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records (AAAI 2024 · similarity 0.86)
• Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks (NeurIPS 2026 · similarity 0.86)
• Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.86)
• ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room (NeurIPS 2026 · similarity 0.86)
• MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey (NeurIPS 2024 · similarity 0.85)