Research map / Detection and segmentation / Open-set and few-shot detection
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UnSeg, which finetunes a SAM-based interactive noise generator via bilevel min-min optimization against a from-scratch surrogate SAM to make images unlearnable for many image segmentation tasks.
Problem
Unauthorized training of image segmentation models on private web images; existing unlearnable example methods were designed for image classification and do not address data efficiency, generation efficiency, and transferability for segmentation.
Approach
Adds learnable noise tokens to SAM's mask decoder to generate universal, mask-prompted error-minimizing noise in one forward pass, trained via bilevel min-min optimization on an interactive segmentation dataset (HQSeg-44K) with a re-initialized MAE-pretrained surrogate SAM, plus epsilon generalization to stabilize training.
Key results
- UnSeg reduces DeepLabV1 and DeepLabV3 mIoU on Pascal VOC 2012 to around 7% (clean 70.6-75.1%).
- On COCO instance segmentation with Mask2Former (Swin-T), UnSeg reduces AP-L to 0.7% and overall AP drop reported as up to 92%.
- UnSeg reduces Mask2Former PQ on Cityscapes by 56.4% (R50), outperforming SynPer, AR and CUDA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate protected training images with UnSeg noise rather than leave them clean | Mask2Former R50 · ADE20K val, panoptic | PQ: 39.7 -> 11.7 | Helped |
| Generate protected training images with UnSeg noise rather than leave them clean | Mask2Former R50 · COCO val2017, panoptic | PQ: 51.9 -> 4.2 | Helped |
| Generate protected training images with UnSeg noise rather than leave them clean | Mask2Former R50 · Cityscapes val, panoptic | PQ: 62.1 -> 5.7 | Helped |
| Generate protected COCO images with UnSeg noise rather than leave them clean | DINO · COCO object detection | AP: 48.7 -> 6.1 | Helped |
| Add epsilon generalization (ablation) | UnSeg · Pascal VOC 2012, semantic | mIoU: 19 -> 7.2 | Helped |
| Add label modification (ablation) | UnSeg · Pascal VOC 2012, semantic | mIoU: 19 -> 56.8 | Hurt |
| Add epsilon generalization and label modification (ablation) | UnSeg · Cityscapes panoptic | PQ: 8.2 -> 5.7 | Helped |
| Use point prompts instead of mask prompts (ablation) | UnSeg · Pascal VOC 2012, semantic | mIoU: 6.2 -> 6.0 | No clear effect |
| Use box prompts instead of mask prompts (ablation) | UnSeg · Pascal VOC 2012, semantic | mIoU: 6.2 -> 6.4 | No clear effect |
| Replace MAE ViT surrogate with supervised ResNet50 (ablation) | UnSeg noise generator · ADE20K panoptic | PQ: 11.7 -> 35.6 | Hurt |
| Replace pretrained SAM generator initialization with random initialization (ablation) | UnSeg noise generator · Pascal VOC 2012, semantic | mIoU: 6.2 -> 4.8 | No clear effect |
Surprising findings
- Class-wise random noise, effective on classification, reduces segmentation mIoU only by 10-20%.
- Mixing UnSeg unlearnable examples with clean data yields performance equal to training on the clean portion only, implying the noise adds little to training.
Limitations
- Protection is weaker on simple-shape classes such as bird and bottle (mIoU 58.6% and 40.4% in multi-class setting).
- Effectiveness depends on the surrogate model choice; a supervised ResNet50 surrogate significantly weakens the generator.
- Mixing unlearnable and clean data can sometimes hurt performance on clean data.
Experimental setup
Datasets: HQSeg-44K Pascal VOC 2012 ADE20K COCO Cityscapes DIS COIFT HRSOD ThinObject WHU NWPU SSDD
Benchmarks: Pascal VOC 2012 ADE20K COCO Cityscapes HQSeg-44K DIS COIFT HRSOD ThinObject WHU NWPU SSDD
Metrics: mIoU PQ AP mAP IoU
Baselines: Random noise (class-wise) Random noise (sample-wise) SynPer AR CUDA JPEG Adversarial training DDC-AT
Builds on: unlearnable examples Segment Anything Model SynPer HQSeg-44K
Closest papers by meaning
- Unsupervised Universal Image Segmentation (CVPR 2024 · similarity 0.88)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.88)
- How Far Are We from True Unlearnability? (ICLR 2025 · similarity 0.86)
- Unsegment Anything by Simulating Deformation (CVPR 2024 · similarity 0.85)
- Detecting and Corrupting Convolution-based Unlearnable Examples (AAAI 2025 · similarity 0.85)
- Universal Segmentation at Arbitrary Granularity with Language Instruction (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.