Research map / Detection and segmentation / Tracking
VideoTrack: Learning To Track Objects via Video Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace tubelet (3D) embedding with separated frame embedding for search and template frames (ablation) | VideoTrack temporal backbone · GOT-10k val | AO: +6.0 (pattern I), +7.5 (pattern VI) | Helped |
| Use space-time positional embedding instead of none (ablation) | VideoTrack · GOT-10k val | AO: +1.2 (pattern I) vs no PE | Helped |
| Use VideoSwin-style 3D shifted window attention instead of vanilla attention (ablation) | VideoTrack · GOT-10k val | AO: -2.1 vs vanilla (69.2 vs 71.3) | Hurt |
| Use trajectory attention instead of vanilla attention (ablation) | VideoTrack · GOT-10k val | AO: +0.2 vs vanilla | No clear effect |
| Use MAE pretraining instead of ImageNet-1k or VideoMAE (ablation) | VideoTrack · GOT-10k val | AO: +2.9 vs ImageNet-1k (73.3 vs 70.4); +9.2 vs VideoMAE | Helped |
| Use VideoMAE pretraining instead of image pretraining (ablation) | VideoTrack · GOT-10k val | AO: -6.3 vs ImageNet-1k (64.1 vs 70.4) | Hurt |
| Add foreground label maps to template frames in the video backbone (ablation) | VideoTrack · GOT-10k val | AO: -6.3 (65.3 vs 71.6) | Hurt |
| Dual-template mechanism (pattern VI) instead of joint space-time attention (pattern I) with more input frames (ablation) | VideoTrack on LaSOT · LaSOT | AO: +0.3 best AO; 9.4% fewer FLOPs and +16.2 FPS at similar frames | Helped |
| Scale number of intermediate templates from 1 to 2 in pattern I (ablation) | VideoTrack pattern I · GOT-10k val | AO: 73.1 to 73.4 at T=2, then drops to 70.3 at T=4 | Mixed |
| Dual-template mechanism (pattern VI) vs. pattern I under increasing intermediate template count (ablation) | VideoTrack · GOT-10k val | AO: 72.1 to 72.7 as T goes 1 to 4 (pattern I falls 73.1 to 70.3) | Helped |
| Replace joint space-time attention with temporal window or message token attention in the two-layer structure (ablation) | VideoTrack · GOT-10k val | AO: -4.9 (window, 68.3 at T=3) and -9.9 (message token, 63.2 at T=2) vs pattern I best 73.1 | Hurt |
| VideoTrack versus OSTrack on GOT-10k test | VideoTrack · GOT-10k test | AO: +1.9 over OSTrack | Helped |
Closest papers by meaning
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.88)
- Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better (CVPR 2025 · similarity 0.88)
- TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking (NeurIPS 2026 · similarity 0.88)
- ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · similarity 0.88)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.