Research map / Video / Video segmentation and tracking
Temporally Consistent Object-Centric Learning by Contrasting Slots
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SLOT CONTRAST, a slot-slot temporal contrastive loss (batch-wide negatives) plus learned slot initialization on a DINOv2-feature recurrent slot attention model, improving temporal consistency and video object discovery on MOVi and YouTube-VIS.
Problem
Unsupervised video object-centric learning often lacks long-term temporal consistency of slot representations across frames, especially on real-world videos with occlusions and interactions.
Approach
Add an InfoNCE-style slot-slot contrastive loss that pulls the same slot across consecutive frames together and pushes apart all other slots in the batch, combined with a learned slot initialization and feature reconstruction of frozen DINOv2 patch features.
Key results
- On YouTube-VIS, video FG-ARI 38.0 and mBO 33.7 versus VideoSAURv2's 31.2 and 29.7.
- Per-frame image FG-ARI on MOVi-E of 84.8, state-of-the-art and above motion-mask weakly supervised methods (DIOD 82.2, MoToK 66.7).
- Batch-wide slot contrast improves MOVi-C FG-ARI by +19.6 over feature reconstruction baseline (49.7 to 69.3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add batch slot-slot contrast to feature reconstruction (ablation) | SLOT CONTRAST · MOVi-C | FG-ARI: 49.7 -> 69.3 | Helped |
| Add batch slot-slot contrast to feature reconstruction (ablation) | SLOT CONTRAST · MOVi-C | mBO: 27.4 -> 32.7 | Helped |
| Use intra-video contrast instead of batch slot-slot contrast (ablation) | SLOT CONTRAST · MOVi-C | FG-ARI: 69.3 -> 54.8 | Hurt |
| Add batch slot-slot contrast to feature reconstruction (ablation) | SLOT CONTRAST · MOVi-E | FG-ARI: 79.8 -> 82.9 | Helped |
| Add batch slot-slot contrast to feature reconstruction (ablation) | SLOT CONTRAST · YouTube-VIS | FG-ARI: 35.3 -> 38.0 | Helped |
| Use learned rather than random initialization (ablation) | Feature Reconstruction · MOVi-E | FG-ARI: 71.1 -> 79.8 | Helped |
| Use learned rather than random initialization (ablation) | SLOT CONTRAST · MOVi-C | FG-ARI: 62.9 -> 69.3 | Helped |
| Use learned rather than random initialization (ablation) | SLOT CONTRAST · MOVi-E | FG-ARI: 75.3 -> 82.9 | Helped |
| Use learned rather than random initialization (ablation) | SLOT CONTRAST · YouTube-VIS | FG-ARI: 36.1 -> 38.0 | Helped |
| Use SLOT CONTRAST rather than VideoSAURv2 | Object discovery · YouTube-VIS | FG-ARI: +6.8 | Helped |
| Use SLOT CONTRAST rather than VideoSAURv2 | Object discovery · YouTube-VIS | mBO: +4 | Helped |
| Use SLOT CONTRAST rather than VideoSAUR | Object discovery · MOVi-C | mBO: 38.9 -> 32.7 | Hurt |
Surprising findings
- Intra-video contrast alone can lead to shutting down too many slots in challenging scenes, while batch-wide contrast avoids this.
- On synthetic MOVi datasets the method produces less sharp masks (lower mBO) than VideoSAUR despite higher FG-ARI.
Limitations
- Fixed number of slots set at initialization
- Cannot directly control segmentation granularity of entities
- On synthetic datasets masks are less sharp (lower mBO) than VideoSAUR
Experimental setup
Datasets: MOVi-C MOVi-E YouTube-VIS 2021
Benchmarks: MOVi-C MOVi-E YouTube-VIS 2021
Metrics: FG-ARI mBO image FG-ARI dynamics prediction FG-ARI/mBO
Baselines: SAVi STEVE VideoSAUR VideoSAURv2 SOLV SlotDiffusion LSD DINOSAUR SlotFormer (feature reconstruction) MoToK DIOD
Builds on: Slot Attention SAVi DINOSAUR VideoSAUR SlotFormer InfoNCE
Closest papers by meaning
- Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities (NeurIPS 2023 · similarity 0.93)
- Rethinking Temporal Consistency in Video Object-Centric Learning: From Prediction to Correspondence (ICML 2026 · similarity 0.92)
- Cycle Consistency Driven Object Discovery (ICLR 2024 · similarity 0.88)
- Object-Centric Pretraining via Target Encoder Bootstrapping (ICLR 2025 · similarity 0.88)
- A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning (CVPR 2025 · similarity 0.87)
- Object centric architectures enable efficient causal representation learning (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.