Research map / Detection and segmentation / Weakly supervised detection and segmentation
Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeCo, a single-staged end-to-end WSSS framework that separates co-occurring objects via CAM-tagged image patches and conquers false activation with dual-teacher multi-granularity contrastive learning.
Problem
Co-occurrence of objects in image-level weakly supervised semantic segmentation causes false activation of co-occurring contexts (e.g., boat/water, train/railroad), since image-level labels give limited supervision.
Approach
Decouple co-occurrence in image space by decomposing images into patches tagged with CAM-derived category labels (with similarity-based tag rectification), and in feature space by a dual-teacher-single-student architecture using tag-guided multi-granularity contrast (patch-to-global prototypes and patch-to-patch memory), trained end-to-end without external data.
Key results
- Achieves 74.0% mIoU on PASCAL VOC val and 73.8% on test for segmentation, and 76.5% pseudo-mask mIoU on VOC train, versus ToCo's 71.1/72.2.
- Reaches 46.7% mIoU on MS COCO val without external data, outperforming ToCo (42.3%) by 4.4 points.
- Ablation: removing the global-teacher contrast (w/o LiG) drops VOC val mIoU from 74.0 to 69.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SeCo instead of ToCo | WSSS on PASCAL VOC · PASCAL VOC val | mIoU: 71.1 -> 74.0 | Helped |
| Use SeCo instead of ToCo | WSSS on PASCAL VOC · PASCAL VOC test | mIoU: 72.2 -> 73.8 | Helped |
| Use SeCo instead of CLIP-ES | WSSS on MS COCO · MS COCO val | mIoU: 45.4 -> 46.7 | Helped |
| Add global category-knowledge contrast (LiG) (ablation) | SeCo · PASCAL VOC val | mIoU: 69.1 -> 74.0 | Helped |
| Add local patch-semantic contrast (LiL) (ablation) | SeCo · PASCAL VOC val | mIoU: 70.3 -> 74.0 | Helped |
| Add tag rectification (ablation) | SeCo · PASCAL VOC val | mIoU: 72.4 -> 74.0 | Helped |
| Add tag rectification (ablation) | SeCo · PASCAL VOC val | precision: 82.6 -> 84.0 | Helped |
Limitations
- Tags from CAM seeds remain noisy; co-category patches still sometimes receive wrong tags.
- Fixed patch size and patch count may not suit all objects; adaptive patch size left for future work.
Experimental setup
Datasets: PASCAL VOC 2012 MS COCO 2014
Benchmarks: PASCAL VOC 2012 MS COCO 2014
Metrics: mIoU confusion ratio precision recall
Baselines: AFA ToCo ViT-PCM 1Stage CLIMS L2G W-OoD CDA MCTformer FPR SLRNet CLIP-ES
Builds on: ToCo L2G InfoNCE AFA
Closest papers by meaning
- Boundary-Enhanced Co-Training for Weakly Supervised Semantic Segmentation (CVPR 2023 · similarity 0.91)
- Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholds (CVPR 2022 · similarity 0.90)
- Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation (AAAI 2024 · similarity 0.90)
- Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning (ICLR 2021 · similarity 0.90)
- W2P: Switching from Weak Supervision to Partial Supervision for Semantic Segmentation (AAAI 2024 · similarity 0.89)
- From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.