Research map / Detection and segmentation / Weakly supervised detection and segmentation
Weakly supervised semantic segmentation
Segment from image-level labels using affinity learning, cue mining, and web data.
116 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Weakly supervised detection and segmentation, keyword cluster “weakly supervised · supervised semantic · image level”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 1 of 6 tracked benchmarks.
26 papers in 2023–24 and 20 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 22.6% to 27.8%.
Open questions the data points to
- MS COCO 2014 val (mIoU): best went from 47.1 (up to 2024) to 47.9 in 2025 — POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation (CVPR 2025 · 14 citations)
- Papers report Baseline on PASCAL VOC 2012 val (mIoU) anywhere from 52.5 to 60.6: comparisons against it depend on whose number you take.
- Papers report BESTIE on MS COCO 2017 val (AP) anywhere from 14.2 to 17.7: comparisons against it depend on whose number you take.
No longer reported by papers here: PASCAL VOC 2012 validation (mIoU, last in 2023); PASCAL VOC 2012 test (mean IoU, last in 2020); PASCAL VOC 2012 train (mIoU, last in 2021).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation (CVPR 2016 · 1,154 citations)
- 2017 · Object Region Mining With Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach (CVPR 2017 · 905 citations)
- 2018 · Learning Pixel-Level Semantic Affinity With Image-Level Supervision for Weakly Supervised Semantic Segmentation (CVPR 2018 · 884 citations)
- 2019 · Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations (CVPR 2019 · 635 citations)
- 2020 · Single-Stage Semantic Segmentation From Image Labels (CVPR 2020 · 277 citations)
- 2021 · BoxInst: High-Performance Instance Segmentation With Box Annotations (CVPR 2021 · 331 citations)
- 2022 · Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation (CVPR 2022 · 205 citations)
- 2023 · CLIP Is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentation (CVPR 2023 · 236 citations)
- 2024 · Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation (CVPR 2024 · 54 citations)
- 2025 · POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation (CVPR 2025 · 14 citations)
- 2026 · ReSAM: Refine, Requery, and Reinforce: Self-Prompting Point-Supervised Segmentation for Remote Sensing Images (CVPR 2026 · 1 citation)
Earliest papers here
- ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation (CVPR 2016 · 1,154 citations)
- Learning Transferrable Knowledge for Semantic Segmentation With Deep Convolutional Neural Network (CVPR 2016 · 188 citations)
- How Hard Can It Be? Estimating the Difficulty of Visual Search in an Image (CVPR 2016 · 117 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
PASCAL VOC 2012 val · mIoU (higher is better)
27 papers, 2017–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 58.1 | Weakly Supervised Semantic Segmentation Using Web-Crawled Videos (CVPR 2017 · 150 citations) |
| 2018 | 74.5 | Normalized Cut Loss for Weakly-Supervised CNN Segmentation (CVPR 2018 · 357 citations) |
| 2021 | 79.2 | Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation (CVPR 2021 · 121 citations) |
| 2024 | 81.6 | Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation (CVPR 2024 · 54 citations) |
PASCAL VOC 2012 test · mIoU (higher is better)
25 papers, 2016–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 73.1 | ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation (CVPR 2016 · 1,154 citations) |
| 2021 | 76.9 | Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021 · 274 citations) |
| 2024 | 81.1 | Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation (CVPR 2024 · 54 citations) |
MS COCO 2014 val · mIoU (higher is better)
5 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 43.8 | Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021 · 22 citations) |
| 2022 | 44.2 | L2G: A Simple Local-to-Global Knowledge Transfer Framework for Weakly Supervised Semantic Segmentation (CVPR 2022 · 189 citations) |
| 2024 | 47.1 | Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation (CVPR 2024 · 54 citations) |
| 2025 | 47.9 | POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation (CVPR 2025 · 14 citations) |
PASCAL VOC 2012 validation · mIoU (higher is better)
5 papers, 2016–2023 · not reported since 2023
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 71.3 | ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation (CVPR 2016 · 1,154 citations) |
| 2023 | 77.2 | Progressive Bayesian Inference for Scribble-Supervised Semantic Segmentation (AAAI 2023 · 1 citation) |
PASCAL VOC 2012 test · mean IoU (higher is better)
4 papers, 2016–2020 · not reported since 2020
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 51.2 | Learning Transferrable Knowledge for Semantic Segmentation With Deep Convolutional Neural Network (CVPR 2016 · 188 citations) |
| 2018 | 63.9 | Bootstrapping the Performance of Webly Supervised Semantic Segmentation (CVPR 2018 · 70 citations) |
| 2020 | 64.3 | Single-Stage Semantic Segmentation From Image Labels (CVPR 2020 · 277 citations) |
PASCAL VOC 2012 train · mIoU (higher is better)
4 papers, 2019–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 66.5 | Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations (CVPR 2019 · 635 citations) |
| 2021 | 78.7 | Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation (CVPR 2021 · 121 citations) |
What has been tried
102 papers here report 1,092 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 6%, hurt 19%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add RFM to the decoder | PASCAL VOC 2012 val · mIoU (%) | 68.7 -> 74.9 | Helped | Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation (CVPR 2024 · 54 citations) |
| Add global category-knowledge contrast (LiG) | PASCAL VOC val · mIoU | 69.1 -> 74.0 | Helped | Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation (CVPR 2024 · 43 citations) |
| Add pseudo-label supervision to scribble supervision | ScribbleSup val · mIoU | 66.17 -> 72.13 | Helped | Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label (AAAI 2024 · 17 citations) |
| add point prompting to direct CLIP + SAM | RefCOCO val · mIoU | 26.46 -> 34.19 | Helped | Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentation (CVPR 2024 · 17 citations) |
| Add multi-cluster activation instead of single-cluster activation | PASCAL VOC 2012 train · mIoU (%) | 71.5 -> 72.9 (+1.4%) | Helped | POT: Prototypical Optimal Transport for Weakly Supervised Semantic Segmentation (CVPR 2025 · 14 citations) |
| add monotonicity constraint | Real noisy · Dice | 0.803 -> 0.838 | Helped | MonoBox: Tightness-Free Box-Supervised Polyp Segmentation Using Monotonicity Constraint (AAAI 2025 · 9 citations) |
| Add SPG using point-cloud spatial priors | Waymo val 3D instance segmentation · mAP | 34.18 -> 42.32; 8.14% mAP | Helped | MWSIS: Multimodal Weakly Supervised Instance Segmentation with 2D Box Annotations for Autonomous Driving (AAAI 2024 · 7 citations) |
| Jointly train the pseudo labeler and instance segmentator instead of training them disjointly | S3DIS Area 5 · AP | 45.3 -> 53.4 | Helped | Sketchy Bounding-box Supervision for 3D Instance Segmentation (CVPR 2025 · 6 citations) |
| Add a WRES head without collaboration modules | RefCOCO val · IoU@0.5 | 63.52 -> 62.89 | Hurt | WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation (CVPR 2025 · 6 citations) |
| Add an initial boundary to binary-map connected-region partitioning | CPM17 · PQ | 0.548 -> 0.707 | Helped | Mudslide: A Universal Nuclear Instance Segmentation Method (CVPR 2024 · 6 citations) |
| Add progressive cue conditioning with CRM | RefCOCOg Val (G) · PointM | 51.7 -> 53.3 | Helped | Boosting Weakly Supervised Referring Image Segmentation via Progressive Comprehension (NeurIPS 2024 · 4 citations) |
| Shorten scribbles to one tenth of their original length | scribble-length ablation · mIoU | less than 2 % mIoU | Hurt | Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets (NeurIPS 2024 · 4 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Baseline | PASCAL VOC 2012 val | mIoU | 52.5 – 60.6 | 3 papers, e.g. CVPR 2018 and CVPR 2019 |
| BESTIE | MS COCO 2017 val | AP | 14.2 – 17.7 | 2 papers, e.g. ICLR 2024 and CVPR 2023 |
What reviewers will expect
Datasets and baselines named in the 7 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- BoxInst · 2 of 7 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 46 papers; from OpenAlex and the papers' affiliations).
- Peking University · 3 papers
- Shanghai Jiao Tong University · 2 papers
- University of Chinese Academy of Sciences · 2 papers
- Zhejiang University · 2 papers
- Beijing Jiaotong University · 2 papers
- The Hong Kong Polytechnic University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Weakly supervised detection (97 papers)
- Weakly supervised object localization (54 papers)
- CAM refinement and prototypes (52 papers)
Elsewhere in the topic
- Pseudo-labeling for semi-supervised detection (76 papers)
- Consistency regularization for segmentation (25 papers)
- Semi-supervised instance segmentation (16 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).