Research map / Detection and segmentation / Tracking
OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds OVT-B, a 1,048-category open-vocabulary multi-object tracking benchmark merged from seven video datasets, and proposes OVTrack+, which adds Kalman motion cues to OVTrack's appearance association.
Problem
Open-vocabulary multi-object tracking lacks a large, densely annotated benchmark; the existing OV-TAO-val/test have few categories (mostly base classes) and sparse 1 fps annotation.
Approach
A large benchmark assembled from seven existing video datasets with unified TAO-style annotation, base/novel splits beyond LVIS overlap, and higher annotation frame rates, plus a baseline that fuses Kalman-predicted IoU motion distance with appearance distance in OVTrack.
Key results
- OVTrack+ reaches 47.0 TETA on OVT-B versus 46.1 for OVTrack (All split).
- On OV-TAO-val, OVTrack+ reaches 38.4 TETA versus 36.1 for OVTrack.
- Classification accuracy (ClsA) stays low on OVT-B, at about 9-14 across methods, with novel-class ClsA the lowest.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add motion information to OVTrack association | OVTrack · OVT-B | TETA (All): 46.1 -> 47.0 | Helped |
| Add motion information to OVTrack association | OVTrack · OVT-B | LocA (All): 60.8 -> 62.0 | Helped |
| Add motion information to OVTrack association | OVTrack · OVT-B | AssA (All): 66.1 -> 67.7 | Helped |
| Add motion information to OVTrack association | OVTrack · OVT-B | ClsA (All): 11.5 -> 11.3 | Hurt |
| Add motion information to OVTrack association | OVTrack · OV-TAO-val | TETA (All): 36.1 -> 38.4 | Helped |
| Add motion information to OVTrack association | OVTrack · OV-TAO-val | LocA (All): 53.8 -> 57.5 | Helped |
| Add motion information to OVTrack association | OVTrack · OV-TAO-val | AssA (All): 37.3 -> 40.8 | Helped |
| Add motion information to OVTrack association | OVTrack · OV-TAO-val | ClsA (All): 17.3 -> 16.9 | Hurt |
Surprising findings
- OVTrack+ slightly lowers classification accuracy (ClsA) relative to OVTrack, so adding motion does not help classification.
- Motion-only trackers such as ByteTrack and OC-SORT score far below appearance-based methods on OVT-B in AssA.
Limitations
- Dataset is test-only with no train/test split, so it cannot be used for training.
- Novel-class classification accuracy is very low for all methods, and the paper's TETA metric averages sub-tasks with unequal difficulty.
- Improvements from OVTrack+ over OVTrack are small (about 0.9 TETA on OVT-B).
Experimental setup
Datasets: AnimalTrack GMOT-40 LV-VIS OVIS UVO YouTube-VIS ImageNet-VID
Benchmarks: OVT-B OV-TAO-val
Metrics: TETA LocA AssA ClsA
Baselines: ByteTrack OC-SORT StrongSORT OVTrack
Builds on: OVTrack ViLD TAO TETA DetPro
Closest papers by meaning
- OVTR: End-to-End Open-Vocabulary Multiple Object Tracking with Transformer (ICLR 2025 · similarity 0.93)
- How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection (AAAI 2024 · similarity 0.90)
- VastTrack: Vast Category Visual Object Tracking (NeurIPS 2024 · similarity 0.89)
- OW-OVD: Unified Open World and Open Vocabulary Object Detection (CVPR 2025 · similarity 0.89)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.89)
- COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.