Research map / Architectures and efficiency / Pruning and efficient tuning
Revisiting End-to-End Learning with Slide-level Supervision in Computational Pathology
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits slide-level end-to-end training of CPath encoders, identifies sparse-attention MIL optimization collapse, and proposes ABMILX (multi-head local attention plus global attention-plus refinement) with multi-scale random patch sampling so an E2E-trained ResNet matches or beats pathology foundation models in two-stage setups.
Problem
Two-stage CPath pipelines use frozen pretrained encoders and disjointly optimized MIL aggregators, while slide-level end-to-end training is computationally expensive and underperforms, with sparse-attention MIL suspected of disrupting encoder optimization.
Approach
Identify that sparse attention in ABMIL causes optimization risk in E2E training, and mitigate it with multi-head local attention plus a global correlation-based attention-plus module (ABMILX), combined with cheap multi-scale random patch sampling to keep E2E training under ~10 RTX3090 GPU hours.
Key results
- E2E-trained ResNet-50 with ABMILX reaches 78.83% accuracy on PANDA, versus 62.72% for the best two-stage ResNet-50 and 74.69% for ABMIL on UNI features.
- ABMILX with ResNet-50 achieves 95.17% AUC on TCGA-BRCA subtyping, +5.82 over the best two-stage ResNet-50 result.
- Total E2E training takes about 9 GPU hours for ResNet-18 ABMILX on BRCA (3090), with 1.7 s/slide inference versus 83 s/slide for GigaPath.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train ResNet end-to-end with ABMILX instead of two-stage UNI features and 2DMamba | ResNet-50 · PANDA | accuracy: 76.37 -> 78.83 | Helped |
| Train ResNet end-to-end with ABMILX instead of two-stage GIGAP features and RRTMIL | ResNet-50 · TCGA-BRCA | AUC: 94.82 -> 95.17 | Helped |
| Train ResNet end-to-end with ABMILX instead of UNI features and RRTMIL | ResNet-18 · TCGA-NSCLC | AUC: 97.88 -> 97.09 | Hurt |
| Train ResNet end-to-end with ABMILX instead of UNI features and TransMIL | ResNet-50 · CPTAC-NSCLC | AUC: 85.24 -> 85.19 | No clear effect |
| Train ResNet end-to-end with ABMILX instead of UNI features and ABMIL | ResNet-50 · CPTAC-LUAD | C-index: 53.59 -> 54.00 | Helped |
| Replace ABMIL with ABMILX (ablation) | End-to-end slide-level learning · CAMELYON | Performance: 91.78 -> 95.88 | Helped |
| Add multi-head local attention (ablation) | ABMILX · Computational pathology tasks | performance | Helped |
| Add global attention plus to refine local attention using patch correlations (ablation) | ABMILX with multi-head local attention · Computational pathology tasks | performance | Helped |
| Freeze the encoder rather than fine-tuning it end-to-end (ablation) | ABMILX · Computational pathology tasks | performance | Hurt |
| Use multi-scale random instance sampling rather than vanilla random sampling (ablation) | End-to-end slide-level learning · Computational pathology tasks | performance | Helped |
| Use attention-based sampling rather than vanilla random sampling (ablation) | End-to-end slide-level learning · Computational pathology tasks | performance | No clear effect |
| Use RRTMIL instead of ABMIL for end-to-end aggregation (ablation) | End-to-end slide-level learning · PANDA, TCGA-NSCLC, and survival tasks | performance | Hurt |
Surprising findings
- Sparse-attention ABMIL (and RRTMIL) degrade severely under E2E training on large slides, underperforming even ResNet-50 two-stage baselines, which the paper attributes to optimization collapse.
- Fine-tuning pretrained aggregators from CHIEF and GigaPath did not yield the desired results, whereas E2E training from ImageNet ResNet did.
Limitations
- Results rely on fixed hyperparameters per task and ablations on ResNet-18 only, with large confidence intervals in survival tasks.
- Theoretical analysis is a simplified argument with assumptions (e.g., weak cross-class correlations) that are not rigorously verified.
- Gains on some tasks (e.g., BLCA, CPTAC-NSCLC) are small or absent relative to foundation-model baselines.
Experimental setup
Datasets: PANDA TCGA-NSCLC TCGA-BRCA TCGA-LUAD TCGA-BLCA CAMELYON-16 CAMELYON-17 CPTAC-LUAD CPTAC-LUSC ImageNet-1K
Benchmarks: PANDA TCGA-NSCLC TCGA-BRCA TCGA-LUAD TCGA-BLCA CPTAC-NSCLC CPTAC-LUAD
Metrics: accuracy AUC C-index training time memory inference time
Baselines: ABMIL CLAM TransMIL DSMIL WIKG RRTMIL 2DMamba UNI CHIEF GigaPath C2C FT
Builds on: ABMIL TransMIL CLAM multi-instance learning
Closest papers by meaning
- Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology (CVPR 2024 · similarity 0.88)
- xMIL: Insightful Explanations for Multiple Instance Learning in Histopathology (NeurIPS 2024 · similarity 0.88)
- FOCUS: Knowledge-enhanced Adaptive Visual Compression for Few-shot Whole Slide Image Classification (CVPR 2025 · similarity 0.87)
- Do Multiple Instance Learning Models Transfer? (ICML 2025 · similarity 0.87)
- Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning (CVPR 2025 · similarity 0.87)
- CPath-Omni: A Unified Multimodal Foundation Model for Patch and Whole Slide Image Analysis in Computational Pathology (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.