Research map / Vision-language and multimodal / Cross-modal learning
Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PIBD, a multimodal (WSI + pathway genomics) survival prediction framework that uses a prototype-based variational information bottleneck (PIB) to remove intra-modal redundancy and a prototype-guided disentanglement module (PID) to split common and modality-specific features.
Problem
Multimodal cancer survival prediction from gigapixel pathology WSIs and genomic pathways suffers from redundancy: task-irrelevant intra-modal information and duplicated inter-modal information that suppresses modality-specific cues.
Approach
Replace per-instance information bottleneck with a set of learnable Gaussian prototypes, one per risk band (8 for 4 time intervals x censorship), that select the most similar instances in each bag; then reuse the joint prototypical distribution (product-of-experts) as a guide for extracting modality-common features, while minimizing mutual information (vCLUB) between common and specific features.
Key results
- PIBD achieves overall mean C-index 0.699 across five TCGA cohorts, versus 0.683 for the second-best method SurvPath.
- PIBD reaches 0.736 C-index on BRCA and 0.768 on COADREAD, the best in those cohorts.
- With an ImageNet ResNet50 encoder instead of TCGA-pretrained CTransPath, PIBD achieves overall 0.671 versus 0.653 for SurvPath.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PIBD instead of SurvPath | multimodal cancer survival prediction · Overall across five cancer datasets | C-index: 0.683 -> 0.699 | Helped |
| use PIBD instead of MOTCat | multimodal cancer survival prediction · HNSC | C-index: 0.656 -> 0.640 | Hurt |
| add PIB to average pooling (ablation) | AP · Overall | C-index: 0.638 -> 0.657 | Helped |
| add PIB to TransMIL (ablation) | TransMIL · Overall | C-index: 0.662 -> 0.672 | Helped |
| add PID to PIB(TransMIL) (ablation) | PIB(TransMIL) · Overall | C-index: 0.672 -> 0.699 | Helped |
| reduce the information retention rate from 100% (ablation) | PIBD · Cancer datasets shown in Figure (e)-(f) | C-index | Mixed |
| increase prototype sample number from 1 to 10 (ablation) | PIBD · BLCA | C-index: 0.614 $\pm$ 0.032 -> 0.659 $\pm$ 0.074 | Helped |
| increase prototype sample number from 10 to 50 (ablation) | PIBD · COADREAD | C-index: 0.804 $\pm$ 0.093 -> 0.768 $\pm$ 0.124 | Hurt |
| increase prototype sample number from 50 to 100 (ablation) | PIBD · STAD | C-index: 0.684 $\pm$ 0.035 -> 0.678 $\pm$ 0.060 | Hurt |
Surprising findings
- Intervening on positive prototypes drops C-index below 0.5 in ablations, while randomly removing a negative prototype causes only a slight decline.
Limitations
- Gains over strongest baselines are modest and within large standard deviations across folds (e.g., BRCA 0.736 ± 0.072).
- Relies on a TCGA-pretrained encoder (CTransPath), with possible data contamination; results reported separately with ImageNet ResNet50.
- Hyperparameters chosen by grid search on validation folds, which may introduce selection bias despite mitigation attempts.
Experimental setup
Datasets: TCGA-BRCA TCGA-BLCA TCGA-COADREAD TCGA-HNSC TCGA-STAD MSigDB Hallmarks Reactome
Benchmarks: TCGA-BRCA TCGA-BLCA TCGA-COADREAD TCGA-HNSC TCGA-STAD
Metrics: C-index Kaplan-Meier log-rank p-value
Baselines: ABMIL TransMIL CLAM Porpoise MCAT MOTCat SurvPath MIB
Builds on: variational information bottleneck CLUB MCAT SurvPath MIB
Closest papers by meaning
- Multimodal Prototyping for cancer survival prediction (ICML 2024 · similarity 0.89)
- Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction (CVPR 2024 · similarity 0.88)
- CA-MLIF: Cross-Attention and Multimodal Low-Rank Interaction Fusion Framework for Tumor Prognostic Prediction (AAAI 2025 · similarity 0.87)
- Robust Multimodal Survival Prediction with Conditional Latent Differentiation Variational AutoEncoder (CVPR 2025 · similarity 0.87)
- Distilled Prompt Learning for Incomplete Multimodal Survival Prediction (CVPR 2025 · similarity 0.87)
- When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.