Research map / Video / Video segmentation and tracking
Unsupervised Open-Vocabulary Object Localization in Videos
Ke Fan, Zechen Bai, Tianjun Xiao, Dominik Zietlow, Max Horn, Zixu Zhao, Carl-Johann Simon-Gabriel, Mike Zheng Shou, Francesco Locatello, Bernt Schiele, Thomas Brox, Zheng Zhang and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add name-assisted joint optimization (merge same-label neighboring slots, drop background) (ablation) | Ours with patch-based CLIP · ImageNet-VID | mAP50: 5.20 -> 29.23 | Improved |
| add name-assisted joint optimization (ablation) | Ours with patch-based CLIP · YouTube-VIS | mAP50: 5.96 -> 35.19 | Improved |
| use patch-based CLIP instead of vanilla CLIP under joint optimization (ablation) | Ours · ImageNet-VID | mAP50: 3.04 -> 29.23 | Improved |
| use vanilla CLIP with joint optimization (ablation) | Ours · ImageNet-VID | CorLoc: 42.99 -> 15.65 | Hurt |
| use video backbone (VideoMAE) with spatio-temporal video slots instead of per-frame slots (ablation) | Ours · ImageNet-VID | mAP50: 25.77 -> 29.23 | Improved |
| use spatio-temporal video slots with DINO image backbone (ablation) | Ours · ImageNet-VID | mAP50: 28.75 -> 28.40 | No clear effect |
| replace pixel-reconstruction slot methods with feature reconstruction (DINOSAUR) | Slot-Attention/SAVi · ImageNet-VID | CorLoc: 16.23 (SAVi) -> 48.04 (DINOSAUR) | Improved |
| add joint optimization to raise single-object slot share (ablation) | Ours · ImageNet-VID | Single-object slot share: 8.87% -> 49.33% | Improved |
| add synonyms to the text vocabulary (ablation) | Ours · ImageNet-VID/YouTube-VIS | localization and labeling | No clear effect |
Closest papers by meaning
- Dense Video Object Captioning from Disjoint Supervision (ICLR 2025 · similarity 0.89)
- Grounding Everything: Emerging Localization Properties in Vision-Language Transformers (CVPR 2024 · similarity 0.88)
- Learning Object State Changes in Videos: An Open-World Perspective (CVPR 2024 · similarity 0.88)
- Transferable Video Moment Localization by Moment-Guided Query Prompting (AAAI 2024 · similarity 0.88)
- RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations (CVPR 2025 · similarity 0.87)
- OW-VISCapTor: Abstractors for Open-World Video Instance Segmentation and Captioning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
