Research map / Vision-language and multimodal / Multimodal LLMs

MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MLLM-HWSI, a hierarchical whole-slide-image MLLM that aligns cell, patch, region and WSI features with pathology language via hierarchical contrastive and cross-scale consistency losses, then instruction-tunes a Qwen2.5-7B LLM for VQA, report and caption generation.

Problem

Existing computational pathology MLLMs compress an entire WSI into a single global embedding, losing fine-grained spatial semantics and failing to mirror how pathologists integrate evidence across magnifications.

Approach

Decompose each WSI into four scales (cells as words, patches as phrases, regions as sentences, WSI as paragraph), encode each with scale-specific encoders and projectors, align each scale to pathology report text with a hierarchical contrastive loss plus a cross-scale consistency loss, and feed the multi-scale tokens to an LLM.

Key results

• Zero-shot WSI classification average BA of 71.86%, versus 64.56% for TITAN.
• Linear probe average BA of 82.48%, versus 75.68% for TITAN.
• SlideBench-VQA (BCNB) accuracy 68.70% and WSI-VQA accuracy 69.20%, exceeding SlideChat (54.1% and 60.1%).

Limitations

• Adds multi-scale encoders and patch filtering, increasing per-WSI inference time relative to baselines.
• Ablations and benchmark numbers in the paper are internally inconsistent across tables (e.g., the same configuration reports different PANDA scores).

Experimental setup

Datasets: WSI-Bench PANDA EBRAINS WSI-VQA SlideBench-VQA TCGA-Slide-Reports HistGen BRACS UBC-Ocean TCGA-OT IMP-CRC

Benchmarks: PANDA EBRAINS BRACS UBC-Ocean TCGA-OT IMP-CRC WSI-VQA SlideBench-VQA WSI-Bench HistGen TCGA-Slide-Reports SlideBench-Caption

Metrics: balanced accuracy weighted F1 accuracy BLEU ROUGE METEOR Recall@K top-1% accuracy

Baselines: SlideChat WSI-LLaVA TITAN Quilt-LLaVA PRISM HistGen CONCH GPT-4V

Builds on: LLaVA CONCH CellViT HIPT Qwen2.5

Closest papers by meaning

• SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding (CVPR 2025 · similarity 0.93)
• PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.92)
• PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology (AAAI 2026 · similarity 0.92)
• PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.91)
• CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.90)
• FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.89)