Research map / Detection and segmentation / Weakly supervised detection and segmentation
Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Feature Self-Reinforcement (FSR), a single-stage ViT-based WSSS method that masks CAM-uncertain patch tokens for self-distilled recovery and enforces consistency on confident regions, reaching 75.7% val mIoU on PASCAL VOC 2012.
Problem
Weakly supervised semantic segmentation with only image-level labels yields CAMs that focus on discriminative regions, producing poor object boundaries and misclassified regions, which degrades pseudo labels and segmentation accuracy.
Approach
Adaptively partition patch tokens into uncertain (intermediate CAM score) and confident regions; mask uncertain tokens and recover them via self-distillation from an EMA teacher, and add a class-token consistency loss on confident regions across two augmented views, all in a single-stage ViT framework without saliency models.
Key results
- Achieves 75.7% mIoU on PASCAL VOC 2012 val and 75.0% on test, outperforming single-stage ToCo (71.1% val) and multi-stage BECO (73.7% val).
- Achieves 45.4% mIoU on MS COCO 2014 val, outperforming single-stage ToCo (42.3%) and slightly exceeding multi-stage MCTformer+ (45.2%).
- Ablation: unc.FSR raises pseudo-label mIoU from 71.1% to 74.4%; adding cer.FSR with MCA aggregation reaches 75.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FSR instead of MCTformer+ | PASCAL VOC 2012 val · PASCAL VOC 2012 val | mIoU: 74.0 -> 75.7 | Helped |
| Use FSR instead of MCTformer+ | MS COCO 2014 val · MS COCO 2014 val | mIoU: 45.2 -> 45.4 | Helped |
| Prioritize uncertain patches for masking instead of random patches (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val | mIoU: random: 73.1, 73.6, 74.1, 74.2, 73.2; uncertain: 74.1, 74.2, 73.9, 74.4, 73.7 | Mixed |
| Add uncertain-feature self-reinforcement (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 71.1 -> 74.4 | Helped |
| Add uncertain-feature self-reinforcement (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val predicted labels | mIoU: 67.9 -> 72.5 | Helped |
| Add certain-feature self-reinforcement with MCA to uncertain-feature self-reinforcement (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 74.4% to 75.7% | Helped |
| Add certain-feature self-reinforcement with MCA to uncertain-feature self-reinforcement (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val predicted labels | mIoU: 72.5% to 73.6% | Helped |
| Replace GAP with MCA for certain-feature aggregation (ablation) | Certain-feature FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 72.3 -> 75.2 | Helped |
| Replace GMP with MCA for certain-feature aggregation (ablation) | Certain-feature FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 71.8 -> 75.2 | Helped |
| Replace default augmentation with GaussianBlur (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 75.7 -> 75.9 ± 0.05 | No clear effect |
| Replace default augmentation with Solarization (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 75.7 -> 75.3 ± 0.12 | No clear effect |
| Replace default augmentation with AutoAugment (ablation) | FSR on PASCAL VOC 2012 val · PASCAL VOC 2012 val pseudo labels | mIoU: 75.7 -> 74.8 ± 0.09 | Hurt |
Surprising findings
- Synchronizing the teacher encoder with the student (EMA momentum 0.0) works better than a slow-moving teacher encoder.
- Strong augmentations such as AutoAugment slightly reduce performance, suggesting data augmentation has limited impact here.
Limitations
- Single-stage transformer methods historically lag multi-stage ones; gains depend on CAM-derived uncertainty thresholds (0.2, 0.7) and masking ratio tuned on validation.
- Data augmentation ablations show limited effect and strong augmentation hurts performance.
Experimental setup
Datasets: PASCAL VOC 2012 SBD MS COCO 2014 ImageNet
Benchmarks: PASCAL VOC 2012 MS COCO 2014
Metrics: mIoU
Baselines: ToCo AFA 1Stage BECO MCTformer+ L2G RCA ViT-PCM OCR ESOL
Builds on: patch token contrast DINO-style self-distillation masked image modeling DeepLab-LargeFOV ToCo
Closest papers by meaning
- Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholds (CVPR 2022 · similarity 0.92)
- Weakly Supervised Semantic Segmentation via Progressive Confidence Region Expansion (CVPR 2025 · similarity 0.91)
- DuPL: Dual Student with Trustworthy Progressive Learning for Robust Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.90)
- W2P: Switching from Weak Supervision to Partial Supervision for Semantic Segmentation (AAAI 2024 · similarity 0.90)
- Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.90)
- From SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.