Research map / Vision-language and multimodal / Cross-modal learning
CA-MLIF: Cross-Attention and Multimodal Low-Rank Interaction Fusion Framework for Tumor Prognostic Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CA-MLIF, a two-stage end-to-end multimodal fusion model combining bidirectional cross-attention with low-rank multimodal fusion and optimal weight concatenation for cancer survival prediction on pathology-genomic and CT-pathology data.
Problem
Survival prediction for cancer patients from multimodal data (pathology whole-slide images, genomics, CT radiomics, clinical), where prior fusion methods are unidirectional, costly, or not end-to-end.
Approach
An improved bidirectional cross-attention (using global context vectors from the other modality, avoiding the QK^T attention map) produces cross-modal interaction features, which are then fused with the original unimodal features via low-rank multimodal fusion and optimal weight concatenation, trained end-to-end with a Cox loss plus an L1 cross-modal alignment term.
Key results
- On multi-center LUAD, CA-MLIF reaches C-index 0.719 vs 0.693 for CMTA and 0.691 for PG-MLIF, the best of the compared methods.
- On TCGA-GBMLGG, CA-MLIF reaches C-index 0.908 vs 0.895 for PG-MLIF.
- On TCGA-KIRC, CA-MLIF reaches C-index 0.732, a marginal gain over CMTA (0.729) and PG-MLIF (0.728).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CA-MLIF instead of PG-MLIF | TCGA-GBMLGG survival prediction · TCGA-GBMLGG | C-index: 0.8950.007 -> 0.9080.011 | Helped |
| Use CA-MLIF instead of CMTA | TCGA-KIRC survival prediction · TCGA-KIRC | C-index: 0.7290.029 -> 0.7320.028 | Helped |
| Use CA-MLIF instead of Pathomic Fusion | MC LUAD survival prediction · MC LUAD | C-index: 0.6840.033 -> 0.7190.021 | Helped |
| Add the Cross-Attention module (ablation) | CA-MLIF · MC LUAD | C-index: 0.6960.024 -> 0.7190.021 | Helped |
| Add the alignment constraint (ablation) | CA-MLIF · MC LUAD | C-index: 0.7030.031 -> 0.7190.021 | Helped |
| Add the second fusion (ablation) | CA-MLIF · MC LUAD | C-index: 0.7080.027 -> 0.7190.021 | Helped |
| Fuse CT and pathology features instead of using CT features alone | CA-MLIF · MC LUAD | C-index: 0.688 -> 0.719 | Helped |
| Fuse CT and pathology features instead of using pathology features alone | CA-MLIF · MC LUAD | C-index: 0.667 -> 0.719 | Helped |
Surprising findings
- Gains on KIRC are not significant; the authors attribute this to the small sample size (417 patients).
Limitations
- Performance gain on TCGA-KIRC is marginal, attributed to small sample size.
- Reported results are from a single in-house LUAD dataset with no external validation described.
- Ablation/significance claims rely on C-index comparisons without detailed statistical description.
Experimental setup
Datasets: TCGA-GBMLGG TCGA-KIRC in-house multi-center LUAD (CT, H&E WSI, clinical)
Benchmarks: TCGA-GBMLGG TCGA-KIRC multi-center LUAD
Metrics: C-index
Baselines: GPDBN MCAT Pathomic Fusion MOTCat CMTA PG-MLIF
Builds on: PG-MLIF CMTA Multimodal Factorization / low-rank tensor fusion Cross-attention (Fukui et al.)
Closest papers by meaning
- Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction (ICLR 2024 · similarity 0.87)
- Distilled Prompt Learning for Incomplete Multimodal Survival Prediction (CVPR 2025 · similarity 0.86)
- HEALNet: Multimodal Fusion for Heterogeneous Biomedical Data (NeurIPS 2024 · similarity 0.86)
- When Are Multimodal Predictions Biologically Supported? A Diagnostic Evaluation Framework (NeurIPS 2026 · similarity 0.85)
- Multimodal Prototyping for cancer survival prediction (ICML 2024 · similarity 0.85)
- Interpretable Vision-Language Survival Analysis with Ordinal Inductive Bias for Computational Pathology (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.