Research map / Detection and segmentation / Open-set and few-shot detection

A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning

CVPR 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Object-centric learning theory and video · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically benchmarks pre-trained vision models for robot learning across pre-training methods and data types, then proposes SlotMIM, which learns object-centric representations from non-object-centric data via a semantic bottleneck and cross-view consistency.

Problem

Pre-trained vision models for robot learning are typically MAE trained on ego-centric data, but it is unclear which pre-training method and data source are optimal; DINO/iBOT degrade on non-(single-)object-centric data, limiting their use.

Approach

Object-centric learning from NOC data: iBOT-style patch-level self-distillation with a small prototype count (semantic bottleneck), a cross-view patch consistency loss via ROIAlign, and MoCo-style contrastive learning over slots formed by prototype-assigned patches.

Key results

• SlotMIM pre-trained on 241K samples outperforms prior methods that used over 1M samples, including MVP and VC-1, on visuomotor tasks.
• On ObjectNav, SlotMIM trained on DetSoup (4.0M, ViT-B) reaches 62.0 vs 55.4 for VC-1 (ViT-B), a 6.6% gain.
• DINO and iBOT performance degrades on scene-centric and NOC data, with objectness correlating 0.72 with task success rate.

Measured effects reported in the paper

Surprising findings

• DINO and iBOT show inverse scaling on manipulation tasks when pre-training data is scaled to the million level, while MAE does not.
• Ego-centric data is not uniquely best; SlotMIM on NOC data scales better than ego-centric-trained MAE-based models.

Limitations

• Navigation experiments were run only on two SlotMIM models due to GPU demands.
• Manipulation evaluation uses only 25 demonstrations per task and no proprioception, a restricted setting.

Experimental setup

Datasets: ImageNet COCO+ CC12M Ego4D Open Images Objects365 LVIS ImageNet-21K DetSoup

Benchmarks: Franka Kitchen Meta-World ObjectNav ImageNav Pascal VOC 2012 segmentation ADE20K COCO object detection COCO instance segmentation ImageNet k-NN

Metrics: success rate Jaccard index mIoU AP box AP mask k-NN accuracy SPL-style success

Baselines: MAE DINO iBOT BEiT SplitMask MVP VC-1 V-Cond

Builds on: iBOT DINO MAE MoCo v3 Slot Attention

Closest papers by meaning

• Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.88)
• Object-Centric Pretraining via Target Encoder Bootstrapping (ICLR 2025 · similarity 0.88)
• Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.87)
• Temporally Consistent Object-Centric Learning by Contrasting Slots (CVPR 2025 · similarity 0.87)
• Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers (NeurIPS 2024 · similarity 0.86)
• PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.86)