Research map / Detection and segmentation / Open-set and few-shot detection

CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrapping

ICLR 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Object-centric learning theory and video · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CrIBo, an online self-supervised method that enforces cross-image consistency between object-level nearest neighbors (via joint-space clustering and cycle-consistent matching) to improve dense representations for in-context and segmentation tasks.

Problem

Image-level bootstrapping entangles object representations in scene-centric images, and existing dense SSL methods are not aligned with dense nearest-neighbor retrieval at test time for in-context scene understanding.

Approach

Extend global nearest-neighbor bootstrapping to the object level: cluster tokens of both augmented views jointly (with positional cues, overclustering), retrieve object-level nearest neighbors from memory banks, keep only cycle-consistent matches, and apply a cross-image self-distillation loss alongside cross-view object and global losses.

Key results

• On dense k-NN retrieval with ViT-B/16 on IN1K, CrIBo reaches 30.0 mIoU on ADE20K (full data) vs 22.0 for DINO and 5.3 for MAE.
• With ViT-S/16 pretrained on COCO, CrIBo achieves 37.5 mIoU on Pascal Context linear segmentation vs 30.0 for CrOC.
• With ViT-S/16 on IN1K 800 epochs, CrIBo obtains 73.7 mIoU on Pascal VOC linear segmentation vs 68.1 for CrOC.

Measured effects reported in the paper

Surprising findings

• Unsupervised clustering outperformed using COCO ground-truth segmentation labels for object-level clustering.
• Object-level cycle-consistency mainly stabilizes training rather than improving the mean performance, per the ablation.
• YOLO-S object detection finetuning results were relatively insensitive to pretraining scheme, which the authors attribute to the finetuning protocol.

Limitations

• Only ViT architectures are studied and compared; larger ViT-L/H models and larger datasets were not explored due to compute budget.
• Finetuning with Segmenter yields comparable performance across methods, so finetuning may be a poor regime to compare SSL methods.
• Object detection evaluation with YOLO-S is weakly informative, as the authors acknowledge.

Experimental setup

Datasets: COCO ImageNet-1k Pascal VOC 2012 ADE20K Pascal Context COCO-Stuff 164K YTVOS

Benchmarks: Pascal VOC 2012 ADE20K Pascal Context COCO-Stuff 164K COCO

Metrics: mIoU AP^bb AP^bb_50 AP^bb_75 top-1 accuracy

Baselines: SlotCon ORL DINO MAE CrOC TimeT LOCA Hummingbird

Builds on: DINO CrOC Sinkhorn-Knopp clustering ORL Segmenter

Closest papers by meaning

• Object-Centric Pretraining via Target Encoder Bootstrapping (ICLR 2025 · similarity 0.87)
• Self-Supervised Visual Representation Learning from Hierarchical Grouping (NeurIPS 2020 · similarity 0.86)
• Bootstrapping Top-down Information for Self-modulating Slot Attention (NeurIPS 2024 · similarity 0.86)
• CrOC: Cross-View Online Clustering for Dense Visual Representation Learning (CVPR 2023 · similarity 0.86)
• Moving Off-the-Grid: Scene-Grounded Video Representations (NeurIPS 2024 · similarity 0.83)
• BOOTPLACE: Bootstrapped Object Placement with Detection Transformers (CVPR 2025 · similarity 0.83)