Research map / Label-efficient and robust learning / Recognition and long-tail

Contrastive Learning for Clinical Outcome Prediction with Partial Data Sources

ICML 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Dense features and foundation priors · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CLOPPS, a contrastive pretraining framework that learns encoders from multiple non-overlapping EHR data sources so that clinical outcome classifiers can run on only one source at inference.

Problem

Clinical outcome prediction models assume the same data sources are available at training and inference, but in practice some sources (e.g., national claims or registry data) are only available for training, leaving only a partial set at deployment.

Approach

Pretrain per-source attention encoders with a contrastive loss that matches observations from the same sample and time across sources, adds a survival-weighted local window loss and a CPC-style future-information loss, then train a single-source classifier on frozen encoder features plus raw covariates.

Key results

• On Moving MNIST, CLOPPS with an MLP classifier outperforms the MLP baseline across all horizons, with larger gains in AP.
• On the Private dataset with only one source at inference, CLOPPS (RWKV classifier) consistently outperforms all five baselines across horizons M={0.5,1,3,5} and both sources.
• Ablation on Private 1-year mortality: full loss LM+LL+LF raises AUC from 0.7371 (MLP) to 0.7527 (+1.9%) and AP from 0.4041 to 0.4202 (+5.7%).

Measured effects reported in the paper

Surprising findings

• Fine-tuning the encoders jointly with the prediction head gave no additional gains over the two-stage training.

Limitations

• Only tabular data considered; images and text left for future work.
• Only two data sources considered.
• Only a single outcome (mortality) studied; competing risks not explored.

Experimental setup

Datasets: Moving MNIST (synthetic) Private dialysis/USRDS EHR dataset MIMIC-III

Benchmarks: Moving MNIST Private real-world dialysis/USRDS dataset MIMIC-III

Metrics: AUC AP

Baselines: Elastic Net MLP RoFormer OPT RWKV

Builds on: SimCLR Contrastive Predictive Coding RoFormer RWKV OPT

Closest papers by meaning

• Memorize and Rank: Elevating Large Language Models for Clinical Diagnosis Prediction (AAAI 2025 · similarity 0.85)
• Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.84)
• Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.84)
• Toward a Well-Calibrated Discrimination via Survival Outcome-Aware Contrastive Learning (NeurIPS 2024 · similarity 0.83)
• SMART: Towards Pre-trained Missing-Aware Model for Patient Health Status Prediction (NeurIPS 2024 · similarity 0.83)
• ACES: Automatic Cohort Extraction System for Event-Stream Datasets (ICLR 2025 · similarity 0.83)