Research map / Architectures and efficiency / Attention and transformers
Scaling Whole-Slide Pathology Foundation Model Pretraining with Billions Off-the-Shelf Tokens
Honglin Li, Zhongyi Shui, Chenglu Zhu, Lin Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CLS-only MIL with token-retention slide pretraining (UNI-1 features) | ABMIL · BRACS tumor grading | F1: 0.692 -> 0.779 | Improved |
| replace CLS-only MIL with token-retention slide pretraining (UNI-1 features) | ABMIL · TCGA-LUAD TP53 mutation | AUC: 0.701 -> 0.763 | Improved |
| use TokRet spatial-token pretraining instead of UNI-2 with ABMIL | UNI-2 (ABMIL) · TCGA survival (5 cohorts average) | C-index: 0.620 -> 0.716 | Improved |
| swap tile backbone from UNI-1 to Conch-1.5 | PathTokScale · TCGA survival (5 cohorts average) | C-index: 0.716 -> 0.731 | Improved |
| PathTokScale (30k slides) vs TITAN (over 330k slides) | TITAN · BCNB ER prediction | AUC: 0.905 -> 0.898 | Mixed |
| PathTokScale (Conch-1.5) vs TITAN on TCGA-BLCA survival | TITAN · TCGA-BLCA survival | C-index: 0.657 -> 0.655 | No clear effect |
| retain few tokens per tile (K at most 4) instead of CLS (ablation) | CLS-only baseline · BRACS tumor grading | performance | Hurt |
| retain more tokens per tile (K at least 8) via TokRet (ablation) | TokRet · BRACS tumor grading | performance | Improved |
| add TokRet top-20 spatial tokens versus CLS-only (ablation) | CLS-only baseline · BRACS tumor grading | performance | Improved |
| use greedy dissimilarity TokRet instead of attention top-K selection (ablation) | attention Top-K · BRACS tumor grading | performance | Improved |
| use TokRet token selection instead of average pooling of spatial tokens (ablation) | avg-pool · BRACS tumor grading | performance | Improved |
| scale pretraining tokens from CLS (0.1B) to 2B via TokRet (ablation) | PathTokScale · BRACS tumor grading | performance | Improved |
Closest papers by meaning
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.93)
- MLLM-HWSI: A Multimodal Large Language Model for Hierarchical Whole Slide Image Understanding (CVPR 2026 · similarity 0.90)
- PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.89)
- Nonparametric Distribution Matching for Self-Supervised Whole-Slide Image Condensation (NeurIPS 2026 · similarity 0.89)
- Context Matters: Query-aware Dynamic Long Sequence Modeling of Gigapixel Images (ICML 2025 · similarity 0.89)
- FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
