Research map / Federated learning and privacy / Personalized and heterogeneous FL
Point Transformer with Federated Learning for Predicting Breast Cancer HER2 Status from Hematoxylin and Eosin-Stained Whole Slide Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a point-transformer federated learning framework with farthest cosine sampling and dynamic distribution adjustment to predict HER2 status from HE-stained WSIs across four sites.
Problem
Predicting HER2 status from HE-stained whole slide images in a multi-site federated setting, where label imbalance and stain/preparation differences across sites hurt performance, and existing MIL and graph methods poorly capture either local context or long-range dependencies.
Approach
Treat each WSI patch as a point with 2D position and deep features, apply a point transformer for local context, sample points with farthest cosine sampling in feature space to capture distinctive long-range features, and use a dynamic label-distribution adjustment with an auxiliary classifier in federated averaging.
Key results
- PointTransformerDDA+ achieves average AUC 0.816 across four federated sites, versus 0.771 for base PointTransformer and 0.790 for TransMIL.
- Under federated setting, PointTransformerDDA+ is closest to centralized training (0.823) among federated methods; best federated baseline FedProx reaches 0.788.
- On two unseen sites (E and F), PointTransformer+ reaches AUC 0.800 and 0.806.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FCS to DDA (ablation) | PointTransformerDDA · Federated test, average | AUC: 0.793 -> 0.816 | Helped |
| Add FCS to the base federated model (ablation) | PointTransformer · Federated test, average | AUC: 0.771 -> 0.806 | Helped |
| Add FCS (ablation) | Local models · Sites A, B, C, and D | AUC: Site A: 0.639 -> 0.659; Site B: 0.799 -> 0.831; Site C: 0.687 -> 0.711; Site D: 0.728 -> 0.729 | Helped |
| Add DDA and FCS together (ablation) | PointTransformer · Federated test, average | AUC: 0.771 -> 0.816 | Helped |
| Add point data augmentation (ablation) | PointTransformer with position information · Federated test, average | AUC: 0.765 -> 0.771 | Helped |
| Replace position information with 0 (ablation) | PointTransformer · Federated test, average | AUC: 0.765 -> 0.700 | Hurt |
| Replace position information with 1 (ablation) | PointTransformer · Federated test, average | AUC: 0.765 -> 0.709 | Hurt |
| Vary the number of group normalization groups (ablation) | FedGroupNorm · Federated test, average | AUC: 0.595 (2 groups); 0.783 (8 groups); 0.782 (32 groups); 0.532 (MAX) | Mixed |
| Reduce training WSIs to 50% (ablation) | PointTransformerDDA+ · Federated test | AUC: 0.763 vs 0.771 | Hurt |
| Reduce training WSIs to 25% (ablation) | PointTransformerDDA+ · Federated test | AUC: 0.708 vs 0.693 | Helped |
| Use FCS instead of the baseline sampling configuration (ablation) | PointTransformer · Unseen Sites E and F | AUC: Site E: 0.795 -> 0.800; Site F: 0.802 -> 0.806 | Mixed |
| Add DDA to the FCS model | PointTransformer+ · IHC Score 2+ subset, average | AUC: 0.747 -> 0.712 | Hurt |
Surprising findings
- FCS brings little benefit at Site D, attributed to Site D's biopsy WSIs having few points so FPS already covers nearly all patches.
- GroupNorm performance is highly sensitive to group number; instance normalization drops AUC to 0.532.
Limitations
- Internal sites B and C are not publicly available, limiting reproducibility.
- Performance is sensitive to normalization choice and FCS gives little gain for biopsy-type site D.
- Unseen-site results are not specifically targeted by the design.
Experimental setup
Datasets: TCGA-BRCA PanNuke HEROHE Challenge Yale HER2 HER2C West China Hospital internal WSIs
Benchmarks: TCGA-BRCA HEROHE Challenge Yale HER2 HER2C internal sites B and C
Metrics: AUC
Baselines: PointTransformer PointNet++ CLAM-SB DSMIL TransMIL HistFL GraphSAGE Patch-GCN SlideGraph+ FedProx FedGroupNorm FedMGDA
Builds on: Point Transformer PointNet++ FedAvg CLAM TransMIL
Closest papers by meaning
- HistoFS: Non-IID Histopathologic Whole Slide Image Classification via Federated Style Transfer with RoI-Preserving (CVPR 2025 · similarity 0.85)
- Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis (NeurIPS 2024 · similarity 0.84)
- Revisiting End-to-End Learning with Slide-level Supervision in Computational Pathology (NeurIPS 2025 · similarity 0.84)
- FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis (AAAI 2025 · similarity 0.83)
- Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction (CVPR 2024 · similarity 0.83)
- Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.