Research map / Label-efficient and robust learning / Self-supervised and contrastive
Decoupled Spatio-Temporal Consistency Learning for Self-Supervised Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSTrack, a self-supervised visual tracker trained without box annotations via decoupled global-forward/local-backward spatio-temporal consistency and an instance contrastive loss.
Problem
Visual object tracking normally requires costly manual box annotations on video frames, limiting dataset scale and diversity; existing self-supervised trackers struggle to exploit spatio-temporal context and instance correspondence.
Approach
Decouples forward tracking (global search over full frames) from backward tracking (local crops with augmented views) within a shared ViT, so the tracker learns spatial localization and temporal association, and adds an instance contrastive loss that pools target features using predicted masks across augmented views to supervise instance identity without labels.
Key results
- On GOT-10k, SSTrack-384 reaches 72.4 AO versus TADS's 46.7 (about 25.7% gain).
- On LaSOT, SSTrack-384 achieves 65.9 AUC, a 20.4% gain over TADS, while remaining 7.3% behind supervised ODTrack in AUC.
- Ablation on GOT-10k: decoupled training without context lifts AO from 41.8 to 68.6; adding spatio-temporal context gives 70.8 and instance contrastive loss gives 72.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add decoupled training without spatio-temporal context (ablation) | SSTrack · GOT10K | AO: 41.8 -> 68.6 | Helped |
| add spatio-temporal context (ablation) | SSTrack decoupled training · GOT10K | AO: 68.6 -> 70.8 | Helped |
| add instance contrastive loss (ablation) | SSTrack · GOT10K | AO: 70.8 -> 72.4 | Helped |
| use DropMAE instead of MAE pre-training (ablation) | SSTrack · GOT10K | AO: 62.7 -> 68.6 | Helped |
| use shear, blur, and LSJ augmentations (ablation) | SSTrack · GOT10K | AO: 68.6 -> 70.3 | Helped |
| remove shear augmentation (ablation) | SSTrack · GOT10K | AO: 70.3 -> 69.4 | Hurt |
| remove blur augmentation (ablation) | SSTrack · GOT10K | AO: 70.3 -> 69.3 | Hurt |
| remove LSJ augmentation (ablation) | SSTrack · GOT10K | AO: 70.3 -> 69.0 | Hurt |
| remove shear, blur, and LSJ augmentations (ablation) | SSTrack · GOT10K | AO: 70.3 -> 68.6 | Hurt |
| use SSTrack-384 rather than the strongest listed tracker | SSTrack-384 · VOT2018 | Acc: 0.562 -> 0.630 | Helped |
Limitations
- Backward tracking performance depends on the localization accuracy of forward tracking.
- Remains behind fully supervised trackers on several benchmarks (e.g., LaSOT, VOT2020 EAO, TNL2K).
Experimental setup
Datasets: LaSOT GOT-10k TrackingNet COCO
Benchmarks: GOT-10k LaSOT TrackingNet LaSOText TNL2K VOT2020 OTB100 UAV123 VOT2018
Metrics: AO SR0.5 SR0.75 AUC normalized precision precision EAO accuracy robustness
Baselines: TADS S2SiamFC self-SDCT CycleSiam SiamRPN++ OSTrack SeqTrack ODTrack ARTrackV2 TransT
Builds on: OSTrack DropMAE Focal loss GIoU loss MoCo
Closest papers by meaning
- Sequential Fusion Based Multi-Granularity Consistency for Space-Time Transformer Tracking (AAAI 2024 · similarity 0.87)
- Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers (CVPR 2024 · similarity 0.86)
- Self-Supervised Multi-Object Tracking with Path Consistency (CVPR 2024 · similarity 0.85)
- SUTrack: Towards Simple and Unified Single Object Tracking (AAAI 2025 · similarity 0.85)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.85)
- Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.