Research map / Video / Video segmentation and tracking
Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation
Shuangrui Ding, Rui Qian, Haohang Xu, Dahua Lin, Hongkai Xiong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add motion consistency loss on top of semantic loss with entropy weighting (ablation) | BA · YTVIS-19 | FG-ARI: 30.1 -> 43.1 (without w), 44.3 with w | Improved |
| add semantic consistency loss and entropy weighting to untrained block (ablation) | BA · YTVIS-19 | FG-ARI: 23.4 -> 30.1 | Improved |
| add entropy-based importance weighting (ablation) | BA · DAVIS-17 Unsupervised | FG-ARI: 38.1 -> 40.1 | Improved |
| cluster learned spatio-temporal attention instead of frame-wise DINO attention (ablation) | BA · YTVIS-19 | FG-ARI: 30.1 -> 44.3 | Improved |
| cluster attention maps rather than output features (ablation) | BA · YTVIS-19 | FG-ARI: 33.9 -> 44.3 | Improved |
| replace hierarchical clustering with K-means (8 centroids) (ablation) | BA · YTVIS-19 | FG-ARI: 44.3 -> 42.5 | No clear effect |
| use temporal stride 4 instead of 1 in training (ablation) | BA · YTVIS-19 | FG-ARI: 37.4 -> 44.3 | Improved |
| increase frames from 3 to 5 (ablation) | BA · YTVIS-19 | FG-ARI: 44.3 -> 44.8 | No clear effect |
| use self-attention maps as motion representation instead of global correlation (ablation) | BA · YTVIS-19 | FG-ARI: 39.1 -> 44.3 | Improved |
| use DINO ViT-S/8 instead of ViT-S/16 (ablation) | BA · YTVIS-19 | FG-ARI: 42.5 -> 44.3 | Improved |
| swap DINO for DINOv2 backbone (ablation) | BA · YTVIS-19 | FG-ARI: 44.3 -> 44.1 (DINOv2 ViT-S/14) | No clear effect |
| sample only 10% of frames as keys at inference (ablation) | BA · YTVIS-19 | FG-ARI: 44.3 -> 43.5, 2.3x speedup | No clear effect |
Closest papers by meaning
- Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentation (CVPR 2023 · similarity 0.91)
- Mask-Guided Self-Supervised Video Object Segmentation (TPAMI 2026 · similarity 0.90)
- SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentation (CVPR 2021 · similarity 0.90)
- Dynamic in Static: Hybrid Visual Correspondence for Self-Supervised Video Object Segmentation (TPAMI 2026 · similarity 0.90)
- Boosting Video Object Segmentation via Space-Time Correspondence Learning (CVPR 2023 · similarity 0.89)
- Point-VOS: Pointing Up Video Object Segmentation (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
