Research map / Vision-language and multimodal / Human and embodied modeling
Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathology
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VLSA, a vision-language survival analysis framework for WSIs that uses GPT-4o-derived prognostic text priors to guide MIL aggregation, ordinal survival prompts, and an ordinal incidence-function objective, with Shapley-based interpretation.
Problem
Survival analysis from gigapixel histopathology WSIs suffers from scarce patient-level data and weakly supervised MIL, limiting prognostic representation learning.
Approach
Use CONCH's pathology vision-language encoders with language-encoded prognostic priors as queries for instance aggregation, ordinal survival prompts built by interpolating base class prompts, and an ordinal incidence-function prediction regularized by squared EMD, with Shapley values over the priors for interpretation.
Key results
- VLSA achieves average CI 0.6954 and average MAE 25.15 across five TCGA datasets, the best among compared methods (runner-up Patch-GCN CI 0.6726).
- In few-shot settings (1-16 shots), VLSA averages CI 0.6237, outperforming all baselines in the average column.
- Ablation: adding language-encoded prognostic priors raises average CI from 0.6625 (attention) to 0.6912 (prognostic texts) and 0.6954 with fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use language-encoded prognostic priors for instance aggregation instead of attention aggregation (ablation) | VLSA · TCGA five-dataset average | CI: 0.6625 -> 0.6912 | Helped |
| Use language-encoded prognostic priors instead of learnable prototypes for instance aggregation (ablation) | VLSA · TCGA five-dataset average | CI: 0.6566 -> 0.6912 | Helped |
| Fine-tune the prognostic-prior features (ablation) | VLSA with prognostic texts · TCGA five-dataset average | CI: 0.6912 -> 0.6954 | Helped |
| Add ordinal survival prompts without prognostic priors (ablation) | VLSA · TCGA five-dataset average | CI: 0.6482 -> 0.6604 | Helped |
| Add ordinal incidence-function regularization without prognostic priors (ablation) | VLSA · TCGA five-dataset average | MAE: 28.70 -> 26.99 | Helped |
| Add ordinal survival prompts and ordinal incidence-function regularization without prognostic priors (ablation) | VLSA · TCGA five-dataset average | CI: 0.6482 -> 0.6625 | Helped |
| Add ordinal survival prompts with prognostic priors (ablation) | VLSA · TCGA BRCA | CI: 0.6304 -> 0.6643 | Helped |
| Add ordinal incidence-function regularization with prognostic priors (ablation) | VLSA · TCGA five-dataset average | MAE: 26.76 -> 25.23 | Helped |
| Use both ordinal biases with prognostic priors instead of removing both (ablation) | VLSA · TCGA five-dataset average | CI: 0.6863 -> 0.6954 | Helped |
| Use both ordinal biases with prognostic priors instead of removing both (ablation) | VLSA · TCGA five-dataset average | MAE: 26.76 -> 25.15 | Helped |
| Use VLSA rather than Patch-GCN | CONCH-based WSI survival analysis · TCGA five-dataset average | CI: 0.6726 -> 0.6954 | Helped |
| Use VLSA rather than Patch-GCN | CONCH-based WSI survival analysis · TCGA five-dataset average | MAE: 26.70 -> 25.15 | Helped |
Surprising findings
- Original CONCH encoders with zero-shot MI-Zero adaptation achieve average CI of only 0.54 without supervised fine-tuning, especially on GBMLGG (0.3842).
Limitations
- Gains in CI over vision-only or other VL baselines are modest on some datasets (e.g., ordinal prompt gains marginal when priors are used).
- Experiments limited to five TCGA cohorts with fewer than 1,000 patients each.
- Prognostic priors generated by GPT-4o and manually filtered, introducing manual curation.
Experimental setup
Datasets: TCGA-BLCA TCGA-BRCA TCGA-GBMLGG TCGA-LUAD TCGA-UCEC
Benchmarks: TCGA-BLCA TCGA-BRCA TCGA-GBMLGG TCGA-LUAD TCGA-UCEC
Metrics: concordance index mean absolute error D-calibration count few-shot CI
Baselines: ABMIL TransMIL ILRA R2T-MIL DeepAttnMISL Patch-GCN MI-Zero ABMIL_Prompt CoOp OrdinalCLIP MI-FewShot
Builds on: CoOp OrdinalCLIP CLAM DeepHit CONCH
Closest papers by meaning
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.88)
- Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation (CVPR 2025 · similarity 0.88)
- FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.88)
- Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction (CVPR 2024 · similarity 0.87)
- Multimodal Prototyping for cancer survival prediction (ICML 2024 · similarity 0.86)
- From Representation Space to Prognostic Insights: Whole Slide Image Generation with Hierarchical Diffusion Model for Survival Prediction (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.