Research map / Label-efficient and robust learning / Self-supervised and contrastive
Time-to-Event Pretraining for 3D Medical Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes time-to-event pretraining that continues pretraining 3D CT encoders on 8,192 EHR-derived time-to-event tasks, improving prognostic prediction without hurting diagnostic classification.
Problem
Self-supervised pretraining for 3D medical imaging captures static structure but fails to link imaging biomarkers to long-term future health outcomes, because supervision is restricted to the same visit or narrow time windows and right-censoring is mishandled.
Approach
Use longitudinal EHR-derived time-to-event labels (thousands of ontology-selected medical code tasks, with censoring) as continued-pretraining supervision for 3D CT encoders via a piecewise exponential neural network loss, then adapt with a frozen encoder and DeepSurv or logistic head.
Key results
- TTE-pretrained models improve prognostic binary AUROC over base models by about 22.6% on average across three architectures (abstract reports 23.7%).
- Averaged Harrell's C-index gain of 29.4% on 8 INSPECT TTE tasks per the abstract; SwinUNETR TTE reaches 0.738 on mortality.
- TTE pretraining yields statistically indistinguishable diagnostic RSPECT AUROCs for most tasks versus other pretraining approaches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add time-to-event pretraining with future EHR labels | 3D image encoders · INSPECT prognostic binary classification | AUROC: 0.128 | Helped |
| Add time-to-event pretraining with future EHR labels | 3D image encoders · INSPECT prognostic TTE | Harrell's C-index: 0.166 | Helped |
| Add future EHR time-to-event labels rather than visit-level labels (ablation) | INSPECT prognostic TTE · INSPECT | Harrell's C-index: 0.038 | Helped |
| Add future EHR time-to-event labels rather than multitask labels | INSPECT prognostic TTE · INSPECT | Harrell's C-index: 0.093 | Helped |
| Add future EHR time-to-event labels rather than visit-level labels (ablation) | INSPECT prognostic binary classification · INSPECT | AUROC: 0.092 | Helped |
| Add time-to-event pretraining | 3D image encoders · RSPECT diagnostic classification | AUROC | No clear effect |
| Add time-to-event pretraining | 3D image encoders · RSPECT RV/LV ≥1 diagnostic classification | AUROC | Helped |
| Replace frozen-encoder linear probing with full-parameter fine-tuning (ablation) | TTE-pretrained DenseNet-121 · INSPECT 12 month PH | AUROC: 0.689 -> 0.631 | Hurt |
Surprising findings
- Single-task full fine-tuning was often no better than linear probing on TTE-pretrained encoders, and sometimes worse.
- Merlin's pretrained ResNet-152 performed similarly to the base weight-inflated ResNet-152 on these tasks.
Limitations
- Only 3D encoders evaluated with frozen backbones and lightweight heads; no fine-tuning or zero/few-shot analysis.
- Pretraining dataset is small (18,945 CT scans) and single modality (chest CT).
- Evaluated only on pulmonary-related tasks from a single institution's EHR (INSPECT).
Experimental setup
Datasets: INSPECT RSPECT Athena OHDSI Vocabularies
Benchmarks: INSPECT RSPECT
Metrics: AUROC Harrell's C-index time-dependent C-statistics integrated Brier score
Baselines: base (no continued pretraining) MTL multitask supervised pretraining visit-level supervised pretraining Merlin ResNet-152
Builds on: Merlin DeepSurv piecewise exponential neural network
Closest papers by meaning
- Autoregressive Sequence Modeling for 3D Medical Image Representation (AAAI 2025 · similarity 0.89)
- One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models (NeurIPS 2026 · similarity 0.87)
- How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks? (ICLR 2024 · similarity 0.87)
- Knowledge Transfer Scaling Laws for 3D Medical Imaging (NeurIPS 2026 · similarity 0.86)
- Multimodal Disease Progression Modeling via Spatiotemporal Disentanglement and Multiscale Alignment (NeurIPS 2025 · similarity 0.86)
- Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.