Research map / Label-efficient and robust learning / Recognition and long-tail
THUNDER: Tile-level Histopathology image UNDERstanding benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces THUNDER, an open-source tile-level benchmark that compares 23 pathology and natural-image foundation models across 16 datasets on classification, segmentation, feature-space, calibration and adversarial robustness tasks.
Problem
Comparing the growing number of digital pathology foundation models is hard because existing benchmarks focus on slide-level MIL or linear probing, lack open-source implementations, and ignore feature-space properties, uncertainty and robustness.
Approach
A patch/tile-level, open-source, dynamic benchmark that isolates encoder representations from slide-level aggregation and adds feature-space alignment (mutual kNN), transformation invariance, calibration (ECE/MCE/ACE/TACE) and PGD adversarial robustness evaluations alongside standard downstream probes.
Key results
- Across tasks, uni2h ranks first in the rank-sum global ranking (rank sum 16), followed by virchow2 (27) and midnight (31); the best vision-language model keep reaches 5th rank.
- Histopathology-specific models generally outperform natural-image models; e.g., uni2h has the best kNN F1 (81.7) and linear probing F1 (83.9) among the listed models.
- Adversarial PGD attacks with epsilon 35e-3 cause strong F1 drops, while vision-language models are more affected than pathology vision models at smaller epsilon.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LoRA adaptation to UNI (ablation) | UNI linear probe · MHIST | linear probing F1-score: +2.3 | Helped |
| Add LoRA adaptation to UNI2-H (ablation) | UNI2-H linear probe · MHIST | linear probing F1-score: +4.9 | Helped |
| Add LoRA adaptation to KEEP (ablation) | KEEP linear probe · MHIST | linear probing F1-score: +7.6 | Helped |
| Add LoRA adaptation across the evaluated models (ablation) | UNI, UNI2-H, Virchow, Virchow2, KEEP, and CONCH linear probes · BRACS | linear probing F1-score: +4.6, -1.3, +2.8, +1.0, -1.3, +2.7 | Mixed |
| Use pathology-pretrained models rather than natural-image-pretrained models | Foundation models · 12 classification datasets | knn F1-score | Helped |
| Use pathology-pretrained models rather than natural-image-pretrained models | Foundation models · 4 segmentation datasets | Dice score | Helped |
| Increase the number of few-shot support samples | Few-shot classifiers using foundation-model embeddings · Classification datasets | F1-score | Helped |
| Train LoRA adapters | Foundation-model feature spaces · BRACS and MHIST | Mutual knn alignment | Hurt |
| Increase the PGD adversarial perturbation budget | Foundation-model linear probes · Classification test sets | F1-score | Hurt |
Surprising findings
- Higher discriminative performance does not always translate to better calibration; e.g., uni2h is not among the best calibrated models.
- Feature-space alignment between models tends to decrease during LoRA adaptation, even for initially well-aligned models.
Limitations
- Only H&E-stained data is covered; IHC and other stains are not supported.
- Datasets are well-studied and may be saturated, and may carry inherent collection biases.
- Calibration results are conditioned on the linear probe classifier and do not study how to combine foundation models.
Experimental setup
Datasets: BACH BRACS BreakHis Camelyon17 WILDS PatchCamelyon CRC-100k MHIST TCGA CRC-MSI CCRCC ESCA TCGA TILS TCGA Uniform
Benchmarks: BACH BRACS BreakHis Camelyon17 WILDS PatchCamelyon CRC-100k MHIST TCGA CRC-MSI CCRCC ESCA TCGA TILS TCGA Uniform
Metrics: accuracy balanced accuracy F1-score Dice score Jaccard index ECE MCE ACE TACE F1 drop under PGD attack
Baselines: UNI UNI2-H Virchow Virchow2 CONCH TITAN KEEP MUSK PLIP QuiltNet CLIP DINOv2
Builds on: DINOv2 LoRA SimpleShot Segmenter PGD
Closest papers by meaning
- STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics (NeurIPS 2024 · similarity 0.87)
- PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.85)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.85)
- Dynamic Entity-Masked Graph Diffusion Model for Histopathology Image Representation Learning (AAAI 2025 · similarity 0.84)
- Sequential Attention-based Sampling for Histopathological Analysis (NeurIPS 2025 · similarity 0.84)
- Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.