Research map / Probabilistic and scientific modeling / Molecules and proteins

One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models

NeurIPS 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Neural point-process models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ORA, a marked time-to-event pretraining objective for structured EHR foundation models that jointly models event timing and numerical values via discretized code-specific likelihood, outperforming NTP, TPP, SurvivEHR and ICareFM across classification, TTE, and regression on MIMIC-IV and CUMC.

Problem

Next-token prediction and existing time-to-event EHR pretraining objectives fail to jointly model irregular event timing, censoring, and associated numerical values in structured EHR, limiting downstream classification, time-to-event, and regression performance.

Approach

A composite code-specific marked point process likelihood: for each medical code, the first future occurrence's time and value are modeled jointly via a discretized (time × value quantile) probability matrix, with censoring handled by a survival-style term, so that values are marginalized when absent.

Key results

• On MIMIC-IV, ORA achieves average AUROC 0.800 across binary classification and average C-index 0.815 for time-to-event, the best among compared models.
• On CUMC, ORA reaches 0.834 average AUROC for binary classification and 0.755 average C-index for time-to-event.
• ORA outperforms Transformer-NTP and Transformer-TPP in most classification and TTE tasks; SurvivEHR and ICareFM achieve higher R^2 on several regression tasks but degrade on classification and TTE.

Surprising findings

• Specialized MTPP models SurvivEHR and ICareFM achieve the highest regression R^2 yet severely degrade on classification and time-to-event tasks.
• A simple 'most recent value' baseline beats all pretrained FMs on several lab regression tasks (e.g., Platelets and Creatinine R^2 on MIMIC-IV).

Limitations

• Quantile-based discretization may be suboptimal for sharply peaked or multimodal distributions.
• Composite likelihood assumes conditional independence of first future occurrences of different codes given history.
• Does not uniformly win on every regression task; ICareFM and SurvivEHR are stronger on some regressions.

Experimental setup

Datasets: MIMIC-IV CUMC (Columbia University Irving Medical Center EHR)

Benchmarks: MIMIC-IV CUMC

Metrics: AUROC time-dependent C-index R^2 RMSE MAE

Baselines: XGBoost Deephit Context Clue (CC-Llama, CC-Mamba) MOTOR SurvivEHR ICareFM Most Recent value Transformer-NTP Transformer-TPP

Builds on: DeepHit Context Clue MOTOR SurvivEHR ICareFM Mamba

Closest papers by meaning

• Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.91)
• MOTOR: A Time-to-Event Foundation Model For Structured Medical Records (ICLR 2024 · similarity 0.88)
• Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.88)
• XTSFormer: Cross-Temporal-Scale Transformer for Irregular-Time Event Prediction in Clinical Applications (AAAI 2025 · similarity 0.88)
• Time-to-Event Pretraining for 3D Medical Imaging (ICLR 2025 · similarity 0.87)
• Toward Better EHR Reasoning in LLMs: Reinforcement Learning with Expert Attention Guidance (AAAI 2026 · similarity 0.85)