Research map / Architectures and efficiency / Attention and transformers
Sequential Attention-based Sampling for Histopathological Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SASHA, a PPO-trained RL agent that scans low-resolution WSI patch features, zooms into 10-20% of high-resolution patches using a HAFED attention MIL feature distiller and a targeted state updater, matching full-resolution MIL methods at a fraction of the cost.
Problem
Analyzing gigapixel whole-slide histopathology images at full high resolution is computationally and memory infeasible, and existing sparse-sampling RL methods (RLogist) lose accuracy relative to full-resolution MIL models while relying on generic ImageNet features.
Approach
Combine a PPO-trained patch-selection agent with a frozen, label-aware hierarchical multi-branch attention feature distiller (HAFED) and a similarity-gated targeted state update (TSU) so that each zoomed patch updates only correlated patches in the WSI state.
Key results
- On CAMELYON16, SASHA-0.2 reaches accuracy 0.953, AUC 0.979, F1 0.937, versus RLogist-0.2 at accuracy 0.862 and AUC 0.879.
- SASHA-0.1 and SASHA-0.2 cut inference time to about 14 s and 26 s per WSI, an 8x and 4x speedup over the second-fastest model, and give over 16x better WSI compressibility than full-resolution methods.
- Under a matched 10%/20% observation budget with random sampling, ACMIL and DTFD drop substantially (e.g., ACMIL-0.2 accuracy 0.838) while SASHA-0.2 reaches 0.953.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sequential SASHA sampling instead of Zoom-MIL at the 20% observation budget | CAMELYON16 · CAMELYON16 | Accuracy: 0.759$\pm$0.001 -> 0.953$\pm$0.017 | Helped |
| use sequential SASHA sampling instead of HAFED full-resolution analysis | CAMELYON16 · CAMELYON16 | Accuracy: 0.963$\pm$0.008 -> 0.953$\pm$0.017 | Hurt |
| use sequential SASHA sampling instead of HAFED full-resolution analysis | TCGA-NSCLC · TCGA-NSCLC | Accuracy: 0.923$\pm$0.011 -> 0.912$\pm$0.010 | Hurt |
| add the WSI-pretrained ViT instead of ResNet-50 features (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.860 -> 0.964 | Helped |
| add the WSI-pretrained ViT instead of CONCH features (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.930 -> 0.964 | Helped |
| add multi-branch classifier attention instead of a single branch (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.899 -> 0.964 | Helped |
| add targeted state updates instead of global state updates (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.837 -> 0.964 | Helped |
| add TSU updates for similar patches instead of updating only the sampled patch (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.860 -> 0.964 | Helped |
| use a learned RL policy instead of a random policy (ablation) | SASHA-0.2 · CAMELYON16 | Accuracy: 0.516 -> 0.964 | Helped |
| use the learned SASHA sampling strategy instead of random patch sampling with ACMIL | CAMELYON16 · SASHA-0.2 vs ACMIL-0.2 | Accuracy: 0.838$\pm$0.035 -> 0.953$\pm$0.017 | Helped |
| increase the sampling budget from 10% to 20% | SASHA · CAMELYON16 | Accuracy: 0.901$\pm$0.021 -> 0.953$\pm$0.017 | Helped |
Surprising findings
- Calibration (ECE) improves as the sampled fraction grows: SASHA-0.2 has lower ECE than ACMIL and DTFD, which process all patches at high resolution.
- Replacing the RL policy with a random policy caused the largest accuracy drop (0.964 to 0.516) in the ablation study.
Limitations
- Training requires all high-resolution patches, which substantially increases training time.
- Observation budget strongly affects calibration; shorter budgets give worse ECE.
- RLogist results are reported as-is from the original paper because its codebase could not be reproduced, so the comparison is not fully controlled.
Experimental setup
Datasets: CAMELYON16 TCGA-NSCLC CamelyonPlus BRACS
Benchmarks: CAMELYON16 TCGA-NSCLC CamelyonPlus BRACS
Metrics: accuracy AUC F1 inference time WSI compressibility expected calibration error
Baselines: Max Pooling ABMIL CLAM TransMIL DTFD ACMIL RLogist Zoom-MIL
Builds on: RLogist ABMIL ACMIL Zoom-MIL PPO
Closest papers by meaning
- Rotation-Agnostic Image Representation Learning for Digital Pathology (CVPR 2024 · similarity 0.88)
- Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024 · similarity 0.88)
- Memory-Efficient Prompt Tuning for Incremental Histopathology Classification (AAAI 2024 · similarity 0.88)
- Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis (CVPR 2024 · similarity 0.88)
- Nonparametric Distribution Matching for Self-Supervised Whole-Slide Image Condensation (NeurIPS 2026 · similarity 0.88)
- PathVQ: Reforming Computational Pathology Foundation Model for Whole Slide Image Analysis via Vector Quantization (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.