Research map / Label-efficient and robust learning / Semi-supervised learning
Soft Self-labeling and Potts Relaxations for Weakly-supervised Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes soft self-labeling for scribble-supervised semantic segmentation, using an ADM-derived joint loss with soft pseudo-labels, collision cross-entropy, and normalized-quadratic/log Potts relaxations, achieving state-of-the-art results on standard architectures.
Problem
Weakly supervised semantic segmentation from sparse scribbles, where unlabeled pixels need unsupervised Potts/CRF-style losses that gradient descent optimizes poorly and hard pseudo-labels cannot represent class uncertainty.
Approach
Replace hard pseudo-labels with soft auxiliary variables in an ADM-split self-labeling loss, using a collision cross-entropy (dot-product based, symmetric) and a log-normalized quadratic Potts relaxation (collision divergence), optimized with a general GPU-friendly continuous solver.
Key results
- Soft self-labeling with HCCE + PCD on DeepLabV3+ (ResNet101, batch 16) reaches 78.1 mIoU on PASCAL VOC 2012 scribbles, versus 78.9 for full supervision with batch 16.
- With batch 12, HCCE + PCD reaches 77.7 mIoU, exceeding full-supervision DeepLabV3+ at 76.6 mIoU.
- With ViT-linear and no pretraining on scribbles, HCCE + PCD reaches 80.94 mIoU, close to full supervision 81.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use soft self-labeling with HCCE + PCD instead of full-mask supervision | DeepLabV3+ · PASCAL VOC 2012 val | mIoU: 76.6 -> 77.7 | Helped |
| Use soft self-labeling with HCCE + PCD instead of full-mask supervision | DeepLabV3+ · PASCAL VOC 2012 val | mIoU: 76.6 -> 78.1 | Helped |
| Use collision cross-entropy instead of standard or reverse cross-entropy for soft pseudo-labels (ablation) | Soft self-labeling · PASCAL VOC 2012 val | mIoU | Helped |
| Use logarithmic Potts relaxations instead of non-log relaxations (ablation) | Soft self-labeling · PASCAL VOC 2012 val | mIoU: 67.24 -> 67.41; 71.05 -> 71.21 | Helped |
| Use normalized relaxations PNQ or PCD instead of PBL or PQ (ablation) | Soft self-labeling · PASCAL VOC 2012 val | mIoU: 67.24 -> 71.12; 71.05 -> 71.22 | Helped |
| Replace nearest-neighbor Potts neighborhood with dense neighborhoods (ablation) | Soft self-labeling · PASCAL VOC 2012 val, full-length scribbles | mIoU: 71.1 -> 67.9 | Hurt |
| Use soft rather than hard pseudo-labels (ablation) | Potts-based self-labeling · PASCAL VOC 2012 val, full-length scribbles | mIoU: 69.6 -> 71.1; 63.1 -> 67.9 | Helped |
| Use dense rather than nearest-neighbor neighborhoods for direct gradient descent (ablation) | Potts-loss gradient descent · PASCAL VOC 2012 val, full-length scribbles | mIoU: 67.0 -> 69.5 | Helped |
| Remove scribble cross-entropy pretraining (ablation) | HCCE + PCD soft self-labeling on DeepLabV3+ · PASCAL VOC 2012 val | mIoU: 77.7 -> 76.7; 78.1 -> 77.6 | Mixed |
| Increase batch size from 12 to 16 (ablation) | HCCE + PCD without pretraining on DeepLabV3+ · PASCAL VOC 2012 val | mIoU: 76.7 -> 77.6 | Helped |
| Increase batch size from 12 to 16 (ablation) | HCCE + PCD without pretraining on ViT-linear · PASCAL VOC 2012 val | mIoU: 80.8 -> 80.94 | Helped |
| Use quadratic rather than collision-divergence Potts relaxation (ablation) | HCCE soft self-labeling · PASCAL VOC 2012 val | mIoU: 77.7 -> 77.5 | Hurt |
Surprising findings
- Soft self-labeling with 3% scribbles can outperform standard cross-entropy training with full pixel-precise masks in some DeepLabV3+ settings.
- Dense neighborhoods perform worse than nearest neighbors for soft self-labeling (67.9 vs 71.1 mIoU), contrary to gradient descent where larger neighborhoods help.
Limitations
- Evaluated mainly on PASCAL VOC 2012 with scribbles; other datasets use block-wise annotations.
- Hyperparameters (eta, lambda, color bandwidth) tuned separately per variant.
- Pseudo-label solver relies on general gradient descent rather than specialized discrete solvers.
Experimental setup
Datasets: PASCAL VOC 2012 augmented Cityscapes ADE20k Natural Scene dataset
Benchmarks: PASCAL VOC 2012 Cityscapes ADE20k
Metrics: mIoU
Baselines: ScribbleSup DenseCRF loss GridCRF loss NonlocalCRF loss BPG URSS SPML PSI SEMINAR TEL AGMM
Builds on: ScribbleSup DenseCRF loss ADMM splitting DeepLabV3+ random walker
Closest papers by meaning
- Reliability Does Matter: An End-to-End Weakly Supervised Semantic Segmentation Approach (AAAI 2020 · similarity 0.88)
- Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning (ICLR 2021 · similarity 0.88)
- W2P: Switching from Weak Supervision to Partial Supervision for Semantic Segmentation (AAAI 2024 · similarity 0.88)
- WISH: Weakly Supervised Instance Segmentation using Heterogeneous Labels (CVPR 2025 · similarity 0.87)
- Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation (AAAI 2024 · similarity 0.87)
- DuPL: Dual Student with Trustworthy Progressive Learning for Robust Weakly Supervised Semantic Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.