Research map / Label-efficient and robust learning / Knowledge distillation
Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings
Christian Grashei, Christian Brechenmacher, Rao Umer, Jingsong Liu, Carsten Marr, Peter Schüffler, Ewa Szczurek
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add random non-aligned cropping augmentation (ablation) | Pathryoshka-B · 10 pathology benchmarks | average accuracy: 81.6 -> 82.4 | Improved |
| add random cropping augmentation (ablation) | Pathryoshka-B · CAMELYON16 | accuracy: 91.6 -> 95.0 | Improved |
| add random cropping augmentation (ablation) | Pathryoshka-B · MHIST | accuracy: 85.1 -> 84.0 | Hurt |
| add random cropping augmentation (ablation) | Pathryoshka-B · PANDA / UniToPatho | accuracy: 77.0 -> 77.0; 54.9 -> 54.9 | No clear effect |
| distill from multiple teachers instead of one (ablation) | Pathryoshka · 10 pathology benchmarks | median balanced accuracy | Improved |
| multi-teacher nested distillation vs single-teacher H0-mini | Pathryoshka-B · 10 pathology benchmarks | median accuracy: 78.6 -> 85.6 | Improved |
| distill ViT-B student from three large teachers vs UNI2-h | Pathryoshka-B · 10 pathology benchmarks | median accuracy: 84.3 -> 85.6 | Improved |
| reduce model from ViT-B to ViT-S | Pathryoshka · 10 pathology benchmarks | median accuracy: 85.6 -> 80.5 | Hurt |
| add nested embeddings and truncate to dim/16 | Pathryoshka-B · ConSep segmentation | Dice: H0-mini 54.8 vs Pathryoshka-B 62.6 | Improved |
| truncate embeddings to dim/64 (nested vs non-nested) | Pathryoshka-B · CCRCC patch retrieval | Recall@5: UNI2-h 92.4 vs 95.2 | Improved |
| truncate embeddings to dim/64 (k-NN) | Pathryoshka-B · 6 patch benchmarks k-NN | relative accuracy decrease: 9.4% vs 17.7% (UNI2-h) | Improved |
Closest papers by meaning
- Scaling Multi-Teacher Distillation for Digital Pathology (NeurIPS 2026 · similarity 0.91)
- PathAsst: A Generative Foundation AI Assistant towards Artificial General Intelligence of Pathology (AAAI 2024 · similarity 0.86)
- Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology (CVPR 2024 · similarity 0.86)
- PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images? (NeurIPS 2026 · similarity 0.86)
- PathGen-1.6M: 1.6 Million Pathology Image-text Pairs Generation through Multi-agent Collaboration (ICLR 2025 · similarity 0.86)
- Revisiting End-to-End Learning with Slide-level Supervision in Computational Pathology (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
