Research map / Detection and segmentation / Weakly supervised detection and segmentation
P2Seg: Pointly-supervised Segmentation via Mutual Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes P2Seg, a mutual distillation module (S2I and I2S) that couples semantic segmentation and instance information to improve point-supervised instance segmentation on PASCAL VOC and MS COCO.
Problem
Point-level supervised instance segmentation (PSIS) struggles to predict precise object boundaries and separate same-class instances because point annotations give only positional cues and lack contours.
Approach
A Mutual Distillation Module with two branches: Semantic-to-Instance (S2I) uses semantic region boundaries and point annotations to build instance labels and offset maps, and Instance-to-Semantic (I2S) uses instance affinity to refine the class map, trained recurrently so each branch supervises the other.
Key results
- Achieves 55.7 mAP50 on PASCAL VOC 2012 with the S2I+I2S ablation (vs 52.8 for BESTIE).
- Reaches 17.6 AP on MS COCO 2017 val with HRNet-48, vs 14.2 for reproduced BESTIE.
- Ablation: adding I2S raises mAP50 from 53.2 to 55.7 on VOC; ours outperforms BESTIE by 2.8 mAP50 with Mask R-CNN and 2.2 with SOLOv2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add S2I to the BESTIE baseline (ablation) | Point-supervised instance segmentation · PASCAL VOC 2012 | mAP50: 52.8 -> 53.2 | Helped |
| Add I2S to S2I (ablation) | P2Seg · PASCAL VOC 2012 | mAP50: 53.2 -> 55.7 | Helped |
| Use Mask R-CNN rather than the BESTIE Mask R-CNN setup (ablation) | P2Seg pseudo-label segmentor · PASCAL VOC 2012 | mAP50: 52.8 -> 55.6 | Helped |
| Use SOLOv2 rather than the BESTIE SOLOv2 setup (ablation) | P2Seg pseudo-label segmentor · PASCAL VOC 2012 | mAP50: 51.9 -> 54.1 | Helped |
| Set hard pixel ratio to 0.1 rather than 0.2 (ablation) | P2Seg training without a segmentor · PASCAL VOC 2012 | mAP50: 50.8 -> 52.0 | Helped |
| Set hard pixel ratio to 0.4 rather than 0.2 (ablation) | P2Seg training without a segmentor · PASCAL VOC 2012 | mAP50: 50.8 -> 51.5 | Hurt |
| Set hard pixel ratio to 0.8 rather than 0.2 (ablation) | P2Seg training without a segmentor · PASCAL VOC 2012 | mAP50: 50.8 -> 50.9 | Hurt |
Surprising findings
- A lower hard pixel ratio (0.1) gave the best mAP50 in the ablation, with reduced ratios risking overfitting to easy examples, which the authors note as a trade-off.
Limitations
- Relies on SAM or off-the-shelf segmentation maps for initial semantic pseudo-labels in the first stage.
- Reported as a two-stage pipeline with a Mask R-CNN fine-tuning step, adding cost beyond the MDM module itself.
- Reproduced BESTIE baselines are marked as the authors' reimplementation, so comparisons depend on reproduction quality.
Experimental setup
Datasets: PASCAL VOC 2012 SBD MS COCO 2017
Benchmarks: PASCAL VOC 2012 val MS COCO 2017 val
Metrics: mAP50 mAP70 mAP75 AP AP50 AP75 IoU-threshold counts average IoU
Baselines: BESTIE Point2Mask WISE-Net BoxInst IRN BBTP Arun et al.
Builds on: BESTIE HRNet Mask R-CNN SOLOv2
Closest papers by meaning
- Semantic-aware SAM for Point-Prompted Instance Segmentation (CVPR 2024 · similarity 0.90)
- WISH: Weakly Supervised Instance Segmentation using Heterogeneous Labels (CVPR 2025 · similarity 0.88)
- The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models (ICLR 2024 · similarity 0.87)
- Training-Free Dataset Pruning for Instance Segmentation (ICLR 2025 · similarity 0.87)
- Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation (AAAI 2024 · similarity 0.87)
- Extreme Point Supervised Instance Segmentation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.