Research map / Detection and segmentation / Tracking
VastTrack: Vast Category Visual Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VastTrack, a large-scale single-object tracking benchmark with 2,115 categories, 50,610 videos, and language annotations, and evaluates 25 trackers on it.
Problem
Existing visual tracking benchmarks cover too few object categories and videos to train or evaluate generalizable trackers for arbitrary object classes.
Approach
A benchmark with vastly more object categories (2,115), many more videos (50,610), long-tailed class distribution, and both bounding-box and language annotations, evaluated under a hybrid protocol where train and test share classes.
Key results
- SeqTrack achieves the best VastTrack scores with 0.402 PRE, 0.429 NPRE, and 0.396 SUC, versus 0.725 SUC on LaSOT.
- Trackers drop sharply on VastTrack: OSTrack falls from 0.711 SUC on LaSOT to 0.336 on VastTrack.
- Further training on VastTrack improves OSTrack SUC on VastTrack from 0.336 to 0.362 and SiamRPN++ from 0.281 to 0.298.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| further train SiamRPN++ on VastTrack | SiamRPN++ · VastTrack | SUC: 0.281 -> 0.298 | Helped |
| further train SiamRPN++ on VastTrack | SiamRPN++ · LaSOT | SUC: 0.496 -> 0.528 | Helped |
| further train OSTrack on VastTrack | OSTrack · VastTrack | SUC: 0.336 -> 0.362 | Helped |
| further train OSTrack on VastTrack | OSTrack · LaSOT | SUC: 0.711 -> 0.722 | Helped |
| evaluate trackers on VastTrack rather than TrackingNet | SeqTrack · TrackingNet vs VastTrack | SUC: 0.855 -> 0.396 | Hurt |
| evaluate trackers on VastTrack rather than LaSOT | SeqTrack · LaSOT vs VastTrack | SUC: 0.725 -> 0.396 | Hurt |
Surprising findings
- RTS, a non-Transformer tracking-by-segmentation method, outperforms several Transformer trackers such as OSTrack and SwinTrack on VastTrack.
- Trackers that perform well on LaSOT (OSTrack, SimTrack) can rank lower on VastTrack than GRM, RTS, and ToMP.
Limitations
- Test set is relatively small (3,500 videos) and the hybrid protocol shares classes between train and test.
- Videos are short-term (mean about 83 frames), so long-term tracking is not covered.
- Several categories have very few videos, giving a long-tailed distribution.
Experimental setup
Datasets: VastTrack
Benchmarks: VastTrack LaSOT TrackingNet
Metrics: precision normalized precision success
Baselines: SiamFC ATOM SiamRPN++ DiMP SiamBAN SiamCAR PrDiMP Ocean STMTrack TrSiam TransT STARK
Builds on: LaSOT GOT-10k TrackingNet OTB
Closest papers by meaning
- OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking (NeurIPS 2024 · similarity 0.89)
- SUTrack: Towards Simple and Unified Single Object Tracking (AAAI 2025 · similarity 0.89)
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos (CVPR 2024 · similarity 0.87)
- DreamTrack: Dreaming the Future for Multimodal Visual Object Tracking (CVPR 2025 · similarity 0.87)
- SMILEtrack: SiMIlarity LEarning for Occlusion-Aware Multiple Object Tracking (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.