Research map / Detection and segmentation / Weakly supervised detection and segmentation
Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Context Prototype-Aware Learning (CPAL), which uses soft positive context prototypes and feature distribution alignment plus a self-supervised consistency loss to produce more complete CAMs for weakly supervised semantic segmentation.
Problem
Image-level weakly supervised semantic segmentation produces incomplete CAMs; contextual prototypes suffer from knowledge bias between instances and contexts, activating similar or co-occurring categories (e.g., cat vs. dog, train vs. railroad).
Approach
Mine in-class attributes from clustered context prototypes in a support bank, weight each candidate neighbor by a pairwise positiveness score relative to the instance prototype anchor, shift instance features toward dense bank distribution via a learned shift term, and supervise CAM with a prototype-aware CAM consistency loss.
Key results
- CPAL on AMN raises CAM seed mIoU from 62.1 to 65.7 on PASCAL VOC train.
- AMN+CPAL reaches 72.5% mIoU on VOC val and 72.9% on test with DeepLabV2 (image-level supervision).
- Ablation on VOC train: baseline 50.1 mIoU rises to 62.5 with full CPAL (prototype-aware, alignment, self-supervised loss).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add plain context prototype learning (ablation) | Classification baseline · PASCAL VOC 2012 train | mIoU: 50.1 -> 51.2 | Helped |
| Add prototype-aware learning with top-K neighbors and positiveness (ablation) | Vanilla context learning · PASCAL VOC 2012 train | mIoU: +3.3% | Helped |
| Add feature alignment (ablation) | Proto-aware CPAL · PASCAL VOC 2012 train | mIoU: +2.3% | Helped |
| Add self-supervised loss (ablation) | CPAL with feature alignment · PASCAL VOC 2012 train | mIoU: +5.7% | Helped |
| Remove candidate neighbors and positiveness (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 59.2 | Hurt |
| Remove positiveness weighting and use all neighbors (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5% to 60.3% | Hurt |
| Set candidate-neighbor count K to 10 instead of 20 (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 61.3 | Hurt |
| Set candidate-neighbor count K to 50 instead of 20 (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 60.1 | Hurt |
| Use Manhattan distance for positiveness (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 59.6 | Hurt |
| Use Euclidean distance for positiveness (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 58.7 | Hurt |
| Use cosine similarity for positiveness (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62.5 -> 61.9 | Hurt |
| Set seed-mask threshold τ to 0.05, 0.15, or 0.2 instead of 0.1 (ablation) | CPAL · PASCAL VOC 2012 train | mIoU: 62, 61.7, 61.4 vs 62.5 | Mixed |
Surprising findings
- Plain context prototype learning (Vanilla) gives only limited gain (+1.1 mIoU) over the baseline, while adding prototype awareness is needed for most of the improvement.
Limitations
- Relies on hyperparameters such as threshold tau, number of prototypes Np, and top-K neighbors, tuned on VOC train.
- Gains on stronger pipelines such as CLIP-ES are smaller (+1.1 seed mIoU on VOC).
Experimental setup
Datasets: PASCAL VOC 2012 SBD MS COCO 2014
Benchmarks: PASCAL VOC 2012 val PASCAL VOC 2012 test MS COCO 2014 val
Metrics: mIoU
Baselines: SEAM AdvCAM CLIMS SIPE ESOL AEFT PPC ReCAM Mat-Label FPR LPCAM ACR
Builds on: class activation mapping SIPE MCTformer AMN CLIP-ES
Closest papers by meaning
- Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP (AAAI 2025 · similarity 0.87)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.87)
- From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.86)
- Semantic-aware SAM for Point-Prompted Instance Segmentation (CVPR 2024 · similarity 0.86)
- Multi-Label Prototype Visual Spatial Search for Weakly Supervised Semantic Segmentation (CVPR 2025 · similarity 0.86)
- PSDPM: Prototype-based Secondary Discriminative Pixels Mining for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.