Research map / Detection and segmentation / Open-set and few-shot detection
Object-Centric Pretraining via Target Encoder Bootstrapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pretrains slot-attention object-centric models from scratch on COCO by bootstrapping an EMA target encoder with cross-view patch filtering to avoid slot collapse.
Problem
Object-centric models with slot attention rely on frozen pretrained non-object-centric target encoders, which caps performance and prevents scalable pretraining on real-world data; naive EMA updating of the target causes performance drops and slot collapse.
Approach
Train object-centric models from scratch via self-distillation where the target encoder is an EMA of the object-centric encoder (so it acquires object-centric inductive biases), and use cross-view nearest-neighbor patch filtering to supervise only informative patches early in training to prevent slot collapse.
Key results
- Trained on COCO+ (~241k images) from scratch, OCEBO reaches MOVi-E FG-ARI 66.8 and EntitySeg FG-ARI 44.2, comparable to DINOSAUR/DINOv2-based models (e.g., DINOSAUR MOVi-E FG-ARI 71.1).
- Removing cross-view patch filtering causes slot collapse (MOVi-E FG-ARI drops from 54.8 to 27.7 on COCO training).
- Only about 10% of patches satisfy the filtering condition in the first epoch, rising to about 70% by epoch 200.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cross-view patch filtering (ablation) | OCEBO · MOVi-E | FG-ARI: 27.7 -> 54.8 | Helped |
| add cross-view patch filtering (ablation) | OCEBO · EntitySeg | FG-ARI: 31.7 -> 41.5 | Helped |
| add the object-centric training objective (λoc) (ablation) | OCEBO · MOVi-E | FG-ARI: 8.0 -> 54.8 | Helped |
| add the object-centric training objective (λoc) (ablation) | OCEBO · EntitySeg | FG-ARI: 30.1 -> 41.5 | Helped |
| add the mask sharpening stage (ablation) | OCEBO · MOVi-E | FG-ARI: 44.0 -> 54.8 | Helped |
| add the mask sharpening stage (ablation) | OCEBO · EntitySeg | mBO: 12.8 -> 15.3 | Helped |
| train on COCO+ rather than COCO (ablation) | OCEBO · MOVi-E | FG-ARI: 54.8 -> 66.8 | Helped |
| train on COCO+ rather than COCO (ablation) | OCEBO · MOVi-E | mBO: 25.8 -> 22.1 | Hurt |
| train on COCO+ rather than COCO (ablation) | OCEBO · EntitySeg | FG-ARI: 41.5 -> 44.2 | Helped |
| train on COCO+ rather than COCO (ablation) | OCEBO · EntitySeg | mBO: 15.3 -> 16.0 | Helped |
| use all patches with an increasing object-centric loss weight instead of cross-view filtering (ablation) | OCEBO · MOVi-E | FG-ARI: 54.8 -> 49.2 | Hurt |
| use all patches with an increasing object-centric loss weight instead of cross-view filtering (ablation) | OCEBO · EntitySeg | FG-ARI: 41.5 -> 36.9 | Hurt |
Surprising findings
- Pretraining on ImageNet yields much lower object discovery performance than COCO, since ImageNet's simple single-object scenes are unsuitable for object-centric pretraining.
- Simpler heuristics for patch supervision (increasing loss weight or random patch dropping) fail to prevent the collapse that cross-view filtering avoids.
Limitations
- Pretraining dataset size is small; scale-up on larger, object-rich datasets remains open.
- Does not match state-of-the-art numbers on all metrics; trades off FG-ARI vs mBO and lacks SPOT/FT-DINOSAUR components.
- Mask boundaries are blurry without an extra mask sharpening stage using a frozen target.
Experimental setup
Datasets: MS COCO train2017 COCO unlabeled2017 (COCO+) ImageNet
Benchmarks: MOVi-C MOVi-E Pascal VOC EntitySeg Pascal VOC in-context semantic segmentation
Metrics: FG-ARI mBO mIoU
Baselines: DINOSAUR FT-DINOSAUR SlotDiffusion SPOT CrOC SlotCon
Builds on: slot attention DINO FT-DINOSAUR DINOSAUR
Closest papers by meaning
- Bootstrapping Top-down Information for Self-modulating Slot Attention (NeurIPS 2024 · similarity 0.90)
- SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers (CVPR 2024 · similarity 0.88)
- A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning (CVPR 2025 · similarity 0.88)
- Temporally Consistent Object-Centric Learning by Contrasting Slots (CVPR 2025 · similarity 0.88)
- CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrapping (ICLR 2024 · similarity 0.87)
- Grounded Object-Centric Learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.