Research map / Learning theory and algorithms / Prediction and decision losses
Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Conformalized Survival Distribution (CSD), a model-agnostic post-process that adapts conformal regression to right-censored data to shift predicted percentile times, improving D-calibration and KM-calibration without hurting C-index.
Problem
Survival models face a discrimination-calibration trade-off: methods that improve calibration usually degrade the C-index, so a model cannot easily be both well ranked and well calibrated under right-censoring.
Approach
Train a survival model for discrimination only, then post-process its individual survival distributions by discretizing them into percentile times and adjusting these with conformal regression, adapted to censored data via KM-sampling of best-guess event times. Theory shows C-index (median/mean based) is preserved and D-cal/KM-cal are asymptotically exact.
Key results
- Across 76 model-dataset comparisons, CSD matched C-index in 60 cases (ties) and was better in 13 (none significant), while non-CSD was better in only 3 (not significant).
- CSD improved D-cal in 68 of 76 comparisons (35 significant) and KM-cal in 56 of 76 (30 significant).
- CSD improved IBS in 61 of 76 comparisons (14 significant) and MAE-PO in 45 of 76 (4 significant).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CSD post-processing to survival predictions | Seven survival algorithms · 11 clinical datasets (76 comparisons) | D-cal: 68 (35 significantly better) of 76 comparisons | Helped |
| Apply CSD post-processing to survival predictions | Seven survival algorithms · 11 clinical datasets (76 comparisons) | KM-cal: 56 (30 significantly better) of 76 comparisons | Helped |
| Apply CSD post-processing to survival predictions | Seven survival algorithms · 11 clinical datasets (76 comparisons) | C-index: 60 ties; 13 CSD wins; 3 non-CSD wins | No clear effect |
| Apply CSD post-processing to survival predictions | Seven survival algorithms · 11 clinical datasets (76 comparisons) | IBS: 61 (14 significantly better) of 76 comparisons | Helped |
| Apply CSD post-processing to survival predictions | Seven survival algorithms · 11 clinical datasets (76 comparisons) | MAE-PO: 45 (4 significantly better) of 76 comparisons | Helped |
| Use KM-sampling instead of deterministic decensoring methods (ablation) | CSD · Medium to large datasets | Calibration performance | Helped |
| Combine training and validation sets as the conformal set instead of using validation alone (ablation) | CSD · Small or medium datasets | Performance | Helped |
| Use validation set alone as the conformal set (ablation) | CSD · Large datasets | Performance | No clear effect |
| Vary the number of discretized percentiles (ablation) | CSD · 11 clinical datasets | Performance | No clear effect |
| Increase the calibration-loss weight in X-cal | Deep log-normal model · METABRIC and SEER-stomach | C-index: drops to ≈ 0.5 | Hurt |
| Optimize with CRPS instead of likelihood loss | Deep log-normal model · METABRIC and SEER-stomach | D-cal | Mixed |
| Optimize with SFM instead of likelihood loss | Deep log-normal model · METABRIC and SEER-stomach | D-cal | Mixed |
Surprising findings
- Post-hoc conformal adjustment shifts predicted curves along the time axis, so survival probability ordering can change even though median and mean time C-index are preserved.
- Calibration of CoxPH and other ISD models could not consistently match KM-level calibration, so KM-sampling relies on KM as the prior.
Limitations
- KM-sampling depends on KM estimates, which need a sufficient number of events; the authors note it is less advantageous for small datasets (N<1000).
- Conformalization requires storing and computing quantiles over the full conformal set, making batch mode unsupported and computation heavier (about 16x time for KM-sampling).
- Assumes exchangeability and conditional independent censoring, which may be violated in practice.
Experimental setup
Datasets: VALCT DLBCL PBC GBM METABRIC GBSG NACD SUPPORT SEER-brain SEER-liver SEER-stomach
Benchmarks: VALCT DLBCL PBC GBM METABRIC GBSG NACD SUPPORT SEER-brain SEER-liver SEER-stomach
Metrics: C-index D-cal KM-cal integrated Brier score MAE-PO
Baselines: AFT Gradient Boosting DeepSurv N-MTLR DeepHit CoxTime CQRNN CRPS X-cal SFM Kaplan-Meier
Builds on: conformal regression conformalized survival analysis D-calibration KM-calibration
Closest papers by meaning
- Toward Conditional Distribution Calibration in Survival Prediction (NeurIPS 2024 · similarity 0.93)
- Doubly Robust Conformalized Survival Analysis with Right-Censored Data (ICML 2025 · similarity 0.89)
- Toward a Well-Calibrated Discrimination via Survival Outcome-Aware Contrastive Learning (NeurIPS 2024 · similarity 0.86)
- On the Burden of Achieving Fairness in Conformal Prediction (NeurIPS 2026 · similarity 0.85)
- SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inference (NeurIPS 2026 · similarity 0.85)
- SimplexUQ: An Evaluation Framework and Benchmark for Conformal Uncertainty on Simplex-Valued Predictions (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.