Research map / Detection and segmentation / Weakly supervised detection and segmentation
Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PBIP, a prototype-based image prompting framework that clusters training patches into per-class MedCLIP image prototypes and uses contrastive foreground/background similarity losses to produce better CAMs for weakly supervised histopathology segmentation.
Problem
Weakly supervised segmentation of histopathology images from image-level labels yields incomplete CAMs, and text-prompted CLIP-style methods struggle due to inter-class homogeneity and intra-class heterogeneity in tissue images.
Approach
Replace text prompts with image prompts: build a clustered image bank of per-class prototypes encoded by MedCLIP, then apply a contrastive similarity loss between foreground/background pixel features and class-specific prototypes to refine pseudo masks.
Key results
- On BCSS-WSSS, PBIP reaches 69.54% mIoU versus 67.86% for Proto2Seg (the second-best weakly supervised method).
- On LUAD-HistoSeg, PBIP achieves 76.44% mIoU, outperforming Proto2Seg (74.91%) and the MLPS baseline (73.54%).
- Ablation on BCSS-WSSS initial pseudo masks: removing the foreground or background similarity loss drops mIoU from 67.54% to 50.23% and 55.01%, respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PBLP instead of Proto2Seg | Weakly supervised segmentation · BCSS-WSSS | mIoU (%): 67.86±0.9∗ -> 69.54±0.6 | Helped |
| use PBLP instead of Proto2Seg | Weakly supervised segmentation · LUAD-HistoSeg | mIoU (%): 74.91±1.2∗ -> 76.44±0.5 | Helped |
| use PBLP instead of Proto2Seg | Weakly supervised segmentation · GCSS | mIoU (%): 53.23±2.8 -> 55.10±1.3 | Helped |
| use PBLP instead of Proto2Seg | Weakly supervised segmentation · BCSS | mIoU (%): 55.34±2.0 -> 55.28±0.8 | Hurt |
| add foreground and background matching losses rather than using classification loss alone (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 62.08±0.3 -> 67.54±0.6 | Helped |
| add foreground matching loss to classification loss (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 62.08±0.3 -> 55.01±2.8 | Hurt |
| add background matching loss to classification loss (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 62.08±0.3 -> 50.23±2.4 | Hurt |
| add the similarity matching module (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 63.25±0.5 -> 68.01±0.6 | Helped |
| add adaptive thresholding (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 63.25±0.5 -> 68.01±0.6 | Helped |
| replace CLIP with MedCLIP (ablation) | SegFormer image encoder · BCSS initial pseudo masks | mIoU (%): 66.45±0.3 -> 68.01±0.6 | Helped |
| use image-bank clustering with K=3 rather than K=1 (ablation) | PBLP · BCSS-WSSS initial pseudo masks | mIoU (%): 67.03±0.4 -> 68.01±0.6 | Helped |
| increase image-bank cluster count from K=3 to K=4 (ablation) | PBLP · GCSS initial pseudo masks | mIoU (%): 53.21±1.2 -> 53.91±1.0 | Helped |
Surprising findings
- Zero-shot classification with text prompts performs poorly across CLIP, PLIP and MedCLIP on histopathology, while image prompts substantially improve results.
- Simply increasing the number of prototype images does not monotonically help; clustering-selected representative images performed better than using all eligible images.
Limitations
- Proto2Seg comparison relies on reimplementation with patch selection that may differ from the original, possibly inflating its numbers.
- Optimal number of clusters K depends on dataset and must be tuned (K=2 to 4 best).
Experimental setup
Datasets: BCSS-WSSS LUAD-HistoSeg GCSS BCSS
Benchmarks: BCSS-WSSS LUAD-HistoSeg GCSS BCSS
Metrics: mIoU FwIoU bIoU dice
Baselines: HistoSegNet TransWS OEEN MLPS TPRO CLIMS QA-CLIMS Proto2Seg SAM-Path SAM2-Path SSPCL
Builds on: CLIP MedCLIP SegFormer DeepLab-v2 TPRO
Closest papers by meaning
- Weakly Supervised Gland Segmentation with Class Semantic Consistency and Purified Labels Filtration (AAAI 2025 · similarity 0.88)
- Incremental Nuclei Segmentation from Histopathological Images via Future-class Awareness and Compatibility-inspired Distillation (CVPR 2024 · similarity 0.87)
- Sequential Attention-based Sampling for Histopathological Analysis (NeurIPS 2025 · similarity 0.86)
- Memory-Efficient Prompt Tuning for Incremental Histopathology Classification (AAAI 2024 · similarity 0.86)
- MATCH: Multi-faceted Adaptive Topo-Consistency for Semi-Supervised Histopathology Segmentation (NeurIPS 2025 · similarity 0.86)
- MAPLE: Multi-scale Attribute-enhanced Prompt Learning for Few-shot Whole Slide Image Classification (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.