Research map / Detection and segmentation / Open-set and few-shot detection
EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EAGLE, an object-centric unsupervised semantic segmentation method that builds EiCue eigenvector clusters from a color and DINO-feature affinity Laplacian and trains with an object-centric contrastive loss (ObjNCE) over prototypes.
Problem
Unsupervised semantic segmentation with self-supervised ViT patch features lacks explicit object-level semantic encoding, so complex objects with diverse structures are poorly segmented.
Approach
Derive object-level cues (EiCue) by spectral clustering of the graph Laplacian combining color affinity and a learnable semantic similarity matrix from DINO features, then use these cues to define object prototypes and an intra- and inter-image object-centric contrastive loss that distills them into the segmentation head.
Key results
- On COCO-Stuff with ViT-S/8, EAGLE reaches 64.2 unsupervised accuracy and 27.2 mIoU, versus 48.3/24.5 for STEGO and 57.2/24.6 for HP.
- On Cityscapes with ViT-S/8, EAGLE achieves 81.8 unsupervised accuracy and 19.7 mIoU, improving over HP (80.1/18.4).
- On Cityscapes with ViT-B/8, EAGLE gets 79.4 accuracy and 22.1 mIoU versus STEGO's 73.2/21.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EAGLE instead of HP | DINO ViT-S/16 USS · COCO-Stuff | unsupervised Acc.: 54.5 -> 60.1 | Helped |
| Use EAGLE instead of HP | DINO ViT-S/16 USS · COCO-Stuff | linear mIoU: 39.1 -> 42.5 | Helped |
| Use EAGLE instead of HP | DINO ViT-S/8 USS · COCO-Stuff | unsupervised Acc.: 57.2 -> 64.2 | Helped |
| Use EAGLE instead of HP | DINO ViT-S/8 USS · COCO-Stuff | linear mIoU: 42.7 -> 43.9 | Helped |
| Use EAGLE instead of HP | DINO ViT-S/8 USS · Cityscapes | unsupervised mIoU: 18.4 -> 19.7 | Helped |
| Use EAGLE instead of HP | DINO ViT-S/8 USS · Cityscapes | linear Acc.: 91.2 -> 91.2 | No clear effect |
| Use EAGLE instead of HP | DINO ViT-B/8 USS · Cityscapes | unsupervised mIoU: 18.4 -> 22.1 | Helped |
| Use EAGLE instead of HP | DINO ViT-B/8 USS · Cityscapes | unsupervised Acc.: 79.5 -> 79.4 | No clear effect |
| Add object-level contrastive learning to correspondence distillation (ablation) | EAGLE on COCO-Stuff · COCO-Stuff | unsupervised Acc.: 46.9 -> 64.2 | Helped |
| Add object-level contrastive learning to correspondence distillation (ablation) | EAGLE on COCO-Stuff · COCO-Stuff | unsupervised mIoU: 21.8 -> 27.2 | Helped |
| Use object loss without the full ObjNCE configuration (ablation) | EAGLE on COCO-Stuff · COCO-Stuff | unsupervised mIoU: 21.8 -> 25.1 | Helped |
| Replace EiCue with K-means cues (ablation) | EAGLE on COCO-Stuff · COCO-Stuff | unsupervised mIoU: 27.2 -> 17.0 | Hurt |
Surprising findings
- Removing the correspondence distillation loss (Exp. 6 in ablation) drops unsupervised mIoU to 17.0, below the variant without the EiCue/K-means contrastive setup, indicating training stability depends on it.
Limitations
- Relies on a frozen DINO backbone and the STEGO/HP training framework, so gains depend on these pretrained features.
- Mainly evaluated on COCO-Stuff and Cityscapes with 27-class protocol; Potsdam-3 results deferred to supplement.
- Hyperparameters such as eigenvector count k and color affinity constraints are tuned per setup.
Experimental setup
Datasets: COCO-Stuff Cityscapes
Benchmarks: COCO-Stuff Cityscapes Potsdam-3
Metrics: pixel accuracy mIoU
Baselines: STEGO HP PiCIE SlotCon TransFGU IIC MDC DINO
Builds on: STEGO HP DINO Deep Spectral Methods PiCIE
Closest papers by meaning
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.87)
- unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoning (ICML 2025 · similarity 0.86)
- Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.86)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.86)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.86)
- UniDSeg: Unified Cross-Domain 3D Semantic Segmentation via Visual Foundation Models Prior (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.