Research map / Detection and segmentation / Open-set and few-shot detection
CuVLER: Enhanced Unsupervised Object Discoveries through Exhaustive Self-Supervised Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VoteCut, a training-free pseudo-label generator that ensembles NCut eigenvector proposals from multiple DINO/DINOv2 ViTs with pixel voting, and CuVLER, a Cascade Mask R-CNN detector trained on these labels with a soft-target instance loss plus cross-domain self-training.
Problem
Unsupervised object discovery and instance segmentation without human annotations; prior single-model pseudo-label methods (MaskCut/CutLER) yield limited mask quality and lack confidence scores.
Approach
Combine NCut-based mask proposals from multiple self-supervised ViTs (DINO and DINOv2 of different patch sizes) via IoU clustering and pixel-level voting, score each cluster by its size relative to the largest cluster, and train a detector with a score-weighted soft-target loss that also enables self-training in new domains.
Key results
- In-domain ImageNet val, CuVLER with CAD reaches AP 33.2 vs. CutLER 29.2 (box AP); VoteCut without CAD reaches 20.9 AP vs. TokenCut 14.4.
- Zero-shot (trained on ImageNet only), CuVLER improves over CutLER on COCO AP50 from 21.9 to 23.0 and on Clipart AP50 from 37.5 to 41.3.
- On COCO val2017 the component ablation shows mask AP rising from 7.7 (MaskCut CAD) to 9.8 with soft-target loss and 10.4 with the self-training stage.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CuVLER instead of the previous SOTA | zero-shot object detection · COCO, COCO20K, VOC, OpenImages, Clipart, Watercolor, Comic | AP50: +1.1; +1.1; +2.5; +2.3; -0.3; +3.8; +1.8 | Mixed |
| use CuVLER instead of the previous SOTA | zero-shot object detection · COCO, COCO20K, VOC, OpenImages, Clipart, Watercolor, Comic | AP: +0.3; +0.2; +2.1; +1.9; +0.6; +3.3; +2.4 | Helped |
| replace MaskCut proposals with VoteCut proposals (ablation) | CAD training · COCO val2017 | APmask50: 15.8 -> 17.7 | Helped |
| replace MaskCut proposals with VoteCut proposals (ablation) | CAD training · COCO val2017 | APmask: 7.7 -> 9.1 | Helped |
| add soft target loss (ablation) | VoteCut CAD · COCO val2017 | APmask50: 17.7 -> 19.3 | Helped |
| add soft target loss (ablation) | VoteCut CAD · COCO val2017 | APmask: 9.1 -> 9.8 | Helped |
| add self-training stage (ablation) | CuVLER · COCO val2017 | APmask50: 19.3 -> 20.4 | Helped |
| add self-training stage (ablation) | CuVLER · COCO val2017 | APmask: 9.8 -> 10.4 | Helped |
Surprising findings
- Using more ViT models in VoteCut consistently improves results, suggesting complementary information across self-supervised models.
- Cross-domain self-training gives only modest gains on LVIS (up to about 7%) due to the category distribution shift from ImageNet.
Limitations
- Computational cost of running multiple ViT models in VoteCut may be prohibitive in resource-constrained settings.
- Using ImageNet for pseudo-label generation may introduce bias from its simplified domain and low object density.
- Cross-domain self-training gains are modest on long-tailed LVIS.
Experimental setup
Datasets: ImageNet COCO train2017 COCO val2017 COCO 20K VOC OpenImages Clipart Watercolor Comic LVIS
Benchmarks: ImageNet validation COCO val2017 COCO 20K VOC OpenImages Clipart Watercolor Comic LVIS
Metrics: AP AP50 AP75 AR100 class-agnostic COCO metrics
Baselines: TokenCut MaskCut CutLER LOST MaskDistill FreeSOLO
Builds on: CutLER MaskCut TokenCut DINO DINOv2
Closest papers by meaning
- Self-Supervised Transformers for Unsupervised Object Discovery Using Normalized Cut (CVPR 2022 · similarity 0.88)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.87)
- unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning (ICML 2025 · similarity 0.86)
- The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models (ICLR 2024 · similarity 0.85)
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.84)
- HEAP: Unsupervised Object Discovery and Localization with Contrastive Grouping (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.