Research map / Vision-language and multimodal / CLIP and zero-shot

Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation

CVPR 2025 · Vision-language and multimodal › CLIP and zero-shot · approach: Self-distillation for vision-language pretraining · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MR-PLIP, a pathology vision-language model pre-trained on 34M multi-resolution TCGA patch-text pairs (5x-40x) with cross-resolution visual-textual alignment and a multi-resolution text-guided visual alignment loss.

Problem

Single-magnification image-text pre-training in computational pathology VLMs generalizes poorly across resolutions, limiting tasks such as cancer subtyping, tissue phenotyping and survival analysis.

Approach

Builds parent-child multi-resolution visual bags and textual bags (descriptions from Quilt-LLaVA), selects top-k positive keywords per patch via cosine similarity, fuses visual and keyword features with a multi-modal encoder, and aligns text-guided features across parent and child resolutions with a SimSiam-style loss.

Key results

• Zero-shot tile-level classification: MR-PLIP reaches weighted F1 of 0.871 on NCT-CRC and 0.935 on DigestPath, outperforming CPLIP and QuiltNet across most datasets.
• Linear probe on NCT-CRC: MR-PLIP BA 0.965 and F1 0.976, the best among compared foundation models except WILDS-CAM17 where UNI leads.
• Ablation: removing both CVTA and MRTVA losses drops CAM16 zero-shot F1 from 0.664 to 0.567 and NCT-CRC from 0.871 to 0.786.

Measured effects reported in the paper

Surprising findings

• Zero-shot performance of existing VLMs varies substantially across magnification levels, with 20x often best and 5x and 40x often worst.
• Quilt-LLaVA descriptions shrink in content and detail as magnification increases, suggesting loss of contextual information at high magnification.

Limitations

• Relies on textual descriptions generated by Quilt-LLaVA, which are unsupervised and may contain irrelevant keywords.
• Inference is slower than some VLM baselines (4.51 vs 2.70-3.00 minutes per WSI).
• Evaluated only on pathology; extension to other medical imaging modalities is future work.

Experimental setup

Datasets: TCGA NCT-CRC PatchCamelyon BACH MHIST SICAP DigestPath CAMELYON16 CAMELYON17 NSCLC-CPTAC EBRAINS PANDA

Benchmarks: NCT-CRC PatchCamelyon BACH MHIST SICAP DigestPath Databiox CAMELYON16 CAMELYON17 NSCLC-CPTAC EBRAINS RCC-DHMC

Metrics: weighted F1 balanced accuracy

Baselines: CLIP PLIP MI-Zero BioCLIP CONCH QuiltNet CPLIP CTransPath DinoSSLPath UNI CHIEF GigaPath

Builds on: PLIP QuiltNet Quilt-LLaVA UNI SimSiam BLIP

Closest papers by meaning

• CPLIP: Zero-Shot Learning for Histopathology with Comprehensive Vision-Language Alignment (CVPR 2024 · similarity 0.90)
• CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.90)
• PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology (AAAI 2026 · similarity 0.89)
• PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.89)
• PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.89)
• MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding (CVPR 2026 · similarity 0.89)