Research map / Learning theory and algorithms / Prediction and decision losses
SAFER: A Calibrated Risk-Aware Multimodal Recommendation Model for Dynamic Treatment Regimes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAFER, a multimodal (structured EHR + clinical notes) transformer for dynamic treatment regime recommendation on sepsis data, with KL-based uncertainty for deceased-patient label ambiguity and BH-based conformal selection for FDR control.
Problem
Dynamic treatment regimes in sepsis suffer from absent optimal-treatment labels (especially for deceased patients), rely mainly on structured EHR, and lack statistical guarantees on recommendation error rates.
Approach
Fuse EHR time series and BioClinicalBERT note embeddings with self- and cross-attention; estimate per-patient uncertainty as KL divergence between a base predictor and a refined module trained on survivors; use a risk-aware loss; and apply conformal p-values with Benjamini-Hochberg to select recommendations with FDR ≤ α.
Key results
- On MIMIC-III, SAFER reports MI-AUC 0.9407 and MA-AUC 0.8672, versus best baseline LSTM/RETAIN at 0.9257 and 0.8219 MA-AUC (RETAIN).
- SAFER achieves the largest counterfactual mortality reduction in the table: 0.3891 on MIMIC-III and 0.4562 on MIMIC-IV, versus 0.2214 and 0.3782 for the best non-SAFER baselines.
- The ablation shows removing risk-aware fine-tuning (SAFER-F) drops MIMIC-IV Macro-AUC to 0.6851 from 0.8755 and mortality reduction to 0.2315 from 0.4562.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add risk-aware fine-tuning (ablation) | SAFER · MIMIC-III | MA-AUC: 0.7140 -> 0.8672 | Helped |
| add risk-aware fine-tuning (ablation) | SAFER · MIMIC-IV | Mortality reduction: 0.2315 -> 0.4562 | Helped |
| include clinical notes alongside structured EHR (ablation) | SAFER · MIMIC-III | MA-AUC: 0.7651 -> 0.8672 | Helped |
| include clinical notes alongside structured EHR (ablation) | SAFER · MIMIC-IV | MA-AUC: 0.7841 -> 0.8755 | Helped |
| incorporate deceased patients through uncertainty-aware training rather than training only on survivors (ablation) | SAFER · MIMIC-III | Mortality reduction: 0.2982 -> 0.3891 | Helped |
| incorporate deceased patients through uncertainty-aware training rather than training only on survivors (ablation) | SAFER · MIMIC-IV | Mortality reduction: 0.3765 -> 0.4562 | Helped |
Surprising findings
- Naive RL achieves the highest MRR@3 on MIMIC-III (0.8891) yet the lowest classification AUCs, and the paper notes RL methods underperform on classification benchmarks.
Limitations
- Assumes survivor labels are reliable and deceased-patient labels are ambiguous; the theory relies on assumptions (Lipschitz student, bounded score) that are loosely justified.
- Assumes i.i.d. calibration and test data; distribution shift is not addressed beyond a remark on weighted conformal inference.
- Evaluation focuses on the survivor subset and the counterfactual mortality model is an auxiliary LSTM, so value estimates are offline and not clinically validated.
Experimental setup
Datasets: MIMIC-III MIMIC-IV
Benchmarks: MIMIC-III sepsis cohort MIMIC-IV sepsis cohort
Metrics: Micro AUC Macro AUC HR@3 MRR@3 counterfactual mortality reduction FDR power
Baselines: LSTM RETAIN TAHDNet Naive RL SRL-RNN ACIL ISL
Builds on: Benjamini-Hochberg procedure conformal inference BioClinicalBERT
Closest papers by meaning
- Evaluating and Learning Optimal Dynamic Treatment Regimes under Truncation by Death (NeurIPS 2025 · similarity 0.87)
- Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.85)
- Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies (NeurIPS 2025 · similarity 0.84)
- One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models (NeurIPS 2026 · similarity 0.84)
- Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination (ICML 2024 · similarity 0.84)
- Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.