Research map / Graph learning / Graph self-supervised learning
Cell Graph Transformer for Nuclei Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Cell Graph Transformer that tokenizes nodes and edges of nuclei cell graphs with link/token markers, initialized via a GCN-guided topology-aware pretraining, for nuclei classification.
Problem
Nuclei classification in histopathology images is limited by GNN-based cell graph methods that pass messages only along fixed local edges; transformers trained on cell graphs suffer from noisy attention and poor convergence with poorly initialized features.
Approach
Treat nodes and edges of a cell graph as transformer tokens with Laplacian-eigenvector link markers and token markers for learnable adjacency, and pretrain the visual feature extractor (U-Net) with a GCN on the same task so that attention starts from topology-aware features.
Key results
- Ours+Hover-net achieves F_avg 0.558 on PanNuke vs 0.503 for Hover-net and 0.521 for SONNET (second best).
- Ours+Hover-net improves F_avg by 3.4% on NuCLS and 3.2% on Lizard over the second-best methods.
- Topology-aware pretraining (M8) outperforms ImageNet-pretrained UNet with CGToken+Transformer (M4) by 21% F_avg on PanNuke.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CGT with Hover-net segmentation masks instead of Hover-net alone | Hover-net · PanNuke | F_avg: 0.503 -> 0.558 | Helped |
| Use CGT with Hover-net segmentation masks instead of Hover-net alone | Hover-net · NuCLS | F_avg: 0.337 -> 0.377 | Helped |
| Use CGT with Hover-net segmentation masks instead of Hover-net alone | Hover-net · Lizard | F_avg: 0.435 -> 0.477 | Helped |
| Use CGT with MCSPat detection results instead of MCSPat | MCSPat · BRCA-M2C | F_avg: 0.507 -> 0.536 | Helped |
| Use GCN pretraining instead of ImageNet pretraining (ablation) | CGToken+Transformer on PanNuke · PanNuke | F_avg: 0.344 -> 0.558 | Helped |
| Use GCN pretraining instead of linear-classifier pretraining (ablation) | CGToken+Transformer on PanNuke · PanNuke | F_avg: 0.420 -> 0.558 | Helped |
| Use GCN pretraining instead of transformer pretraining (ablation) | CGToken+Transformer on PanNuke · PanNuke | F_avg: 0.484 -> 0.558 | Helped |
| Use CGToken+Transformer instead of a vanilla transformer classifier (ablation) | UNet GCN pretrained on PanNuke · PanNuke | F_avg: 0.512 -> 0.558 | Helped |
| Use CGToken+Transformer instead of a linear classifier (ablation) | UNet linear pretrained on PanNuke · PanNuke | F_avg: 0.341 -> 0.420 | Helped |
| Use CGToken+Transformer instead of a GCN classifier (ablation) | UNet GCN pretrained on PanNuke · PanNuke | F_avg: 0.518 -> 0.558 | Helped |
| Increase transformer layer count from 1 to 4 (ablation) | CGT on PanNuke · PanNuke | F_avg: 0.6%; 0.4%; 1.5% | Helped |
| Increase GCN pretraining edge number from 4 to 8 (ablation) | GCN pretraining on PanNuke · PanNuke | F_avg: 1.2% | Hurt |
Surprising findings
- Simple initialization of the feature extractor fails to train the cell graph transformer; CGToken+Transformer with ImageNet-pretrained UNet reaches only 0.344 F_avg on PanNuke.
Limitations
- Requires an external binary segmentation or detection tool to produce nuclei centroids.
- Gains depend on the segmentation tool quality; Ours+GT results show headroom from accurate centroids.
- Limited to a few transformer layers due to GPU memory constraints.
Experimental setup
Datasets: PanNuke Lizard NuCLS BRCA-M2C
Benchmarks: PanNuke Lizard NuCLS BRCA-M2C
Metrics: F-score per class average F-score F_avg AJI PQ detection F1 F_d
Baselines: Hover-net MCSPatnet SONNET Mask2former NCCD DDOD YOLOX ConvNext-UperNet DINO
Builds on: Hover-net GENConv Laplacian eigendecomposition positional encoding U-Net FPN
Closest papers by meaning
- UniCell: Universal Cell Nucleus Classification via Prompt Learning (AAAI 2024 · similarity 0.85)
- MERGE: Multi-faceted Hierarchical Graph-based GNN for Gene Expression Prediction from Whole Slide Histopathology Images (CVPR 2025 · similarity 0.85)
- Dynamic Entity-Masked Graph Diffusion Model for Histopathology Image Representation Learning (AAAI 2025 · similarity 0.84)
- NLGT: Neighborhood-based and Label-enhanced Graph Transformer Framework for Node Classification (AAAI 2025 · similarity 0.84)
- Multi-modal Topology-embedded Graph Learning for Spatially Resolved Genes Prediction from Pathology Images with Prior Gene Similarity Information (CVPR 2025 · similarity 0.84)
- Higher-Order Cell Tracking Transformer (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.