Research map / Detection and segmentation / Open-set and few-shot detection
HEAP: Unsupervised Object Discovery and Localization with Contrastive Grouping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEAP, a lightweight head on a frozen DINO ViT that groups patches into regions with learnable group tokens and trains them with intra-image grouping, cross-image region clustering, and foreground-background contrastive losses for unsupervised object discovery and saliency detection.
Problem
Unsupervised object discovery and localization relies on patch-level similarity within a single image, which ignores region- and image-level relationships across images and cannot distinguish multiple object classes in one image.
Approach
Learnable image-specific group tokens are cross-attended with frozen self-supervised patch features to form semantic regions; region embeddings are split into foreground/background, trained with a modularity-based intra-image grouping loss, a negative fg-bg contrastive loss, and a rank-weighted inter-image clustering loss, with no external supervision or second-stage self-training.
Key results
- HEAP reaches 77.1 CorLoc on VOC12 object discovery, the best among compared methods (FOUND 76.1).
- On DUTS-TE saliency detection with dCRF, HEAP reaches 0.949 Acc, 0.687 IoU and 0.777 max F-beta.
- On VOC12 semantic segmentation retrieval (21 classes), HEAP surpasses FOUND by 3.3% mIoU for multiple-object retrieval (46.0 vs 42.7).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add inter-image clustering loss (ablation) | HEAP · DUTS-TE | max Fβ: .731 -> .777 | Helped |
| Add entropy regularization (ablation) | HEAP · DUTS-TE | max Fβ: .700 -> .731 | Helped |
| Set group-token count M to 8 rather than 3 or 15 (ablation) | HEAP · DUTS-TE | IoU: .604 / .654 -> .687 | Helped |
| Set inter-image clustering weight α to 0.1 rather than 0.25 or 0.5 (ablation) | HEAP · DUTS-TE | max Fβ: .756 / .762 -> .777 | Helped |
| Apply dense CRF post-processing | HEAP · DUTS-TE | IoU: .644 -> .687 | Helped |
| Use HEAP instead of FOUND (ViT-S/8) | unsupervised object discovery · VOC12 | CorLoc: 76.1 -> 77.1 | Helped |
| Use HEAP instead of FOUND (ViT-S/8) | multiple-object retrieval · VOC12 21cls | mIoU: 42.7 -> 46.0 | Helped |
Surprising findings
- DINOSAUR achieves higher CorLoc on COCO20K (67.2) than HEAP (63.4), though the authors note it needs far heavier training.
Limitations
- Patch-level predictions give coarse segmentation, requiring dense CRF post-processing to refine boundaries.
- Underperforms DINOSAUR on COCO20K object discovery (63.4 vs 67.2 CorLoc).
Experimental setup
Datasets: VOC12 VOC07 COCO20K DUT-OMRON DUTS-TE ECSSD
Benchmarks: VOC12 VOC07 COCO20K DUT-OMRON DUTS-TE ECSSD
Metrics: mIoU CorLoc Acc IoU max F-beta
Baselines: FOUND TokenCut LOST SelfMask FreeSOLO C2AM DINOSAUR MaskContrast
Builds on: DINO FOUND LOST TokenCut GroupViT
Closest papers by meaning
- Unsupervised Object Localization: Observing the Background To Discover Objects (CVPR 2023 · similarity 0.88)
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.85)
- unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning (ICML 2025 · similarity 0.85)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.84)
- CuVLER: Enhanced Unsupervised Object Discoveries through Exhaustive Self-Supervised Transformers (CVPR 2024 · similarity 0.84)
- Integrating Low-Level Visual Cues for Enhanced Unsupervised Semantic Segmentation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.