Research map / Probabilistic and scientific modeling / Variational and generative sampling
Robust Multimodal Survival Prediction with Conditional Latent Differentiation Variational AutoEncoder
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LD-CVAE, a conditional latent differentiation VAE with a variational information bottleneck transformer, to generate functional genomic embeddings from WSIs so survival prediction works when genomic data is missing at test time.
Problem
Multimodal cancer survival prediction from histopathology WSIs and genomics assumes both modalities are available, but genomic data is costly and often missing at test time; generating genomic features from gigapixel WSIs is hard, especially across functionally diverse gene groups.
Approach
Combine a transformer-based variational information bottleneck for compressing WSI bags with a latent differentiation VAE that learns a shared genomic posterior plus function-specific posteriors per gene category, fused via product-of-experts conditioned on pathology and aligned with a Wasserstein-based loss.
Key results
- Overall C-index 0.719 on complete multimodal data, outperforming the best multimodal baseline SurvPath (0.709) and CMTA (0.706).
- With missing genomic data at test time, overall C-index 0.684, versus 0.670 for ShaSpec and 0.669 for a Transformer baseline.
- Ablations show removing VIB-Trans drops complete-modality overall C-index from 0.719 to 0.701; removing the alignment loss drops it to 0.703.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VIB-Trans instead of the TransMIL encoder for WSI representation (ablation) | LD-CVAE · Overall, complete modality | C-index: 0.701 -> 0.719 | Helped |
| Use VIB-Trans instead of the TransMIL encoder for WSI representation (ablation) | LD-CVAE · Overall, missing genomic data | C-index: 0.665 -> 0.684 | Helped |
| Use LD-VAE with function-specific posteriors instead of a standard VAE (ablation) | LD-CVAE · Overall, complete modality | C-index: 0.707 -> 0.719 | Helped |
| Use LD-VAE with function-specific posteriors instead of a standard VAE (ablation) | LD-CVAE · Overall, missing genomic data | C-index: 0.675 -> 0.684 | Helped |
| Add the alignment loss (ablation) | LD-CVAE · Overall, complete modality | C-index: 0.703 -> 0.719 | Helped |
| Add the alignment loss (ablation) | LD-CVAE · Overall, missing genomic data | C-index: 0.669 -> 0.684 | Helped |
Surprising findings
- Generated genomic features give co-attention maps and patient stratification similar to those from genuine genomic data, per the paper's visualizations.
Limitations
- Missing-modality performance (0.684) remains below complete-modality performance (0.719).
- Evaluated only on five TCGA cohorts with genomic data as the missing modality.
Experimental setup
Datasets: TCGA BLCA TCGA BRCA TCGA GBMLGG TCGA LUAD TCGA UCEC
Benchmarks: TCGA BLCA TCGA BRCA TCGA GBMLGG TCGA LUAD TCGA UCEC
Metrics: C-index log-rank p-value (Kaplan-Meier)
Baselines: MLP SNN SNNTrans ABMIL CLAM-SB CLAM-MB TransMIL Porpoise MCAT MOTCat CMTA SurvPath
Builds on: CVAE MCAT VIB MVAE TransMIL
Closest papers by meaning
- Multimodal Prototyping for cancer survival prediction (ICML 2024 · similarity 0.88)
- Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction (ICLR 2024 · similarity 0.87)
- Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction (CVPR 2024 · similarity 0.86)
- Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathology (ICLR 2025 · similarity 0.86)
- Leveraging Tumor Heterogeneity: Heterogeneous Graph Representation Learning for Cancer Survival Prediction in Whole Slide Images (NeurIPS 2024 · similarity 0.85)
- SSC-VAE: Structured Sparse Coding Based Variational Autoencoder for Detail Preserved Image Reconstruction (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.