Research map / Vision-language and multimodal / Cross-modal learning
ModalTune: Fine-Tuning Slide-Level Foundation Models with Multi-Modal Information for Multi-task Learning in Digital Pathology
Vishwesh Ramanathan, Tony Xu, Pushpak Pati, Faruk Ahmed, Maged Goubran, Anne L. Martel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ViT-Adapter-style Modal Adapter instead of LoRA (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.940 -> 0.954 | Improved |
| add ViT-Adapter-style Modal Adapter instead of LoRA (ablation) | ModalTune · TCGA survival | C-index: 0.744 -> 0.750 | Improved |
| add transcriptomics tokens to the Modal Adapter (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.920 -> 0.954 | Improved |
| add transcriptomics tokens to the Modal Adapter (ablation) | ModalTune · TCGA survival | C-index: 0.707 -> 0.750 | Improved |
| add text-embedding multi-task training (ablation) | ModalTune · TCGA survival | C-index: 0.730 -> 0.750 | Improved |
| add text-embedding multi-task training (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.948 -> 0.954 | Improved |
| add text-embedding multi-task training (ablation) | ABMIL (cat) · TCGA subtype | balanced accuracy: 1.6% drop | Hurt |
| add task-specific prompts to the general prompt (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.935 -> 0.954 | Improved |
| add task-specific prompts to the general prompt (ablation) | ModalTune · TCGA survival | C-index: 0.739 -> 0.750 | Improved |
| use CONCH text encoder instead of Llama-3-8B-UltraMedical (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.921 -> 0.954 | Improved |
| add frozen random projector (ablation) | ModalTune · TCGA survival | C-index: 0.730 -> 0.750 | Improved |
| make the projector trainable instead of frozen (ablation) | ModalTune · TCGA subtype | balanced accuracy: 0.954 -> 0.652 | Hurt |
Closest papers by meaning
- Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction (ICLR 2024 · similarity 0.88)
- Modalities Contribute Unequally: Enhancing Medical Multi-modal Learning through Adaptive Modality Token Re-balancing (ICML 2025 · similarity 0.88)
- CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.87)
- Revisiting End-to-End Learning with Slide-level Supervision in Computational Pathology (NeurIPS 2025 · similarity 0.86)
- Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning (CVPR 2025 · similarity 0.86)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
