Research map / Detection and segmentation / Open-set and few-shot detection
Pay Attention to the Foreground in Object-Centric Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a contrastive, teacher-student foreground/background indicator (ViT-B/16 DINO-initialized) to slot attention models such as DINOSAUR and SPOT, plus spectral-clustering region combination at test time, to improve unsupervised object-centric segmentation on MOVi-C, PASCAL VOC and COCO.
Problem
Slot attention methods in unsupervised object-centric learning over-segment complex real-world backgrounds by low-level color/texture cues and neglect detailed foreground object segmentation.
Approach
An unsupervised contrastive foreground/background indicator (pixel-level consistency, stuff-level background consistency, and entropy-based separation losses) is trained once and its features are fused into existing slot attention encoders without retraining the indicator; spectral clustering over slot similarities merges over-segmented regions at test time.
Key results
- On PASCAL VOC, DINOSAUR+FS+RC reaches 45.2 mBOi / 52.9 mBOc vs 44.0 / 51.2 for DINOSAUR; on COCO, SPOT+FS+RC reaches 35.7 mBOi / 45.3 mBOc vs 34.7 / 44.3 for SPOT.
- On MOVi-C, SPOT+FS+RC improves mBOi from 47.0 to 49.0 and DINOSAUR+FS+RC from 42.4 to 46.0.
- Ablation on PASCAL shows removing the entropy separation loss collapses the indicator into one slot (IoU 28.4 without it vs 52.6 with all losses).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the foreground-background indicator fusion stage to DINOSAUR | DINOSAUR · MOVi-C | mBOi: 42.4 -> 45.3 | Helped |
| add the foreground-background indicator fusion stage to DINOSAUR | DINOSAUR · PASCAL VOC 2012 | mBOi: 44.0 -> 45.4 | Helped |
| add the foreground-background indicator fusion stage to DINOSAUR | DINOSAUR · MS COCO 2017 | mBOi: 31.1 -> 31.5 | Helped |
| add the foreground-background indicator fusion stage to SPOT | SPOT · MOVi-C | mBOi: 47.0 -> 48.5 | Helped |
| add the foreground-background indicator fusion stage to SPOT | SPOT · PASCAL VOC 2012 | mBOi: 48.1 -> 49.1 | Helped |
| add the foreground-background indicator fusion stage to SPOT | SPOT · MS COCO 2017 | mBOi: 34.7 -> 35.5 | Helped |
| add region combination after fusion (ablation) | DINOSAUR + FS · PASCAL VOC 2012 | mBOi: 45.4 -> 45.2 | Hurt |
| add region combination after fusion (ablation) | DINOSAUR + FS · MS COCO 2017 | mBOi: 31.5 -> 31.2 | Hurt |
| add region combination after fusion (ablation) | DINOSAUR + FS · PASCAL VOC 2012 | mBOc: 52.5 -> 52.9 | Helped |
| add region combination after fusion (ablation) | SPOT + FS · MS COCO 2017 | mBOi: 35.5 -> 35.7 | Helped |
| add region combination without fusion (ablation) | DINOSAUR · PASCAL VOC 2012 | mBOi: 44.0 -> 43.3 | Hurt |
| add region combination without fusion (ablation) | SPOT · MS COCO 2017 | mBOi: 34.7 -> 33.2 | Hurt |
Surprising findings
- Applying region combination alone to DINOSAUR and SPOT often degrades performance; it only helps when the fusion stage provides a clear foreground/background distinction.
- Pretrained DINO ViT used directly as an indicator gives only a slight gain, while random-noise indicator causes a large drop.
Limitations
- The indicator cannot perfectly separate foreground and background without supervision, so it may introduce noise or incorrect labels.
- Region combination can merge two instances of the same category, reducing mBOi on PASCAL and COCO.
Experimental setup
Datasets: MOVi-C PASCAL VOC 2012 MS COCO 2017
Benchmarks: MOVi-C PASCAL VOC 2012 MS COCO 2017
Metrics: mBOi mBOc mIoU
Baselines: Slot Attention SLASH SLATE SlotDiffusion LSD DINOSAUR SPOT
Builds on: Slot Attention DINOSAUR SPOT DINO
Closest papers by meaning
- Bootstrapping Top-down Information for Self-modulating Slot Attention (NeurIPS 2024 · similarity 0.84)
- Finding Distributed Object-Centric Properties in Self-Supervised Transformers (CVPR 2026 · similarity 0.83)
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.83)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.82)
- OBJECT-UNI: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation (NeurIPS 2026 · similarity 0.81)
- Interpreting Object-level Foundation Models via Visual Precision Search (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.