Research map / Detection and segmentation / Tracking
NetTrack: Tracking Highly Dynamic Objects with a Net
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NetTrack, a multi-object tracker that tracks highly dynamic open-world objects using fine-grained point-level "Nets" for association and grounding-based object-text correspondence, and introduces the BFT bird flock tracking benchmark.
Problem
Multi-object tracking of highly dynamic open-world objects, where deformation, fast motion and occlusion distort box and appearance cues used for association and open-world localization.
Approach
Replace coarse box/appearance association with a fine-grained Net of sampled points of interest tracked by a point tracker; match Nets to detections by the fraction of points falling inside each detection box, combined with IoU; use phrase-grounding pre-training (Grounding DINO) for fine-grained object-text localization.
Key results
- On BFT with Grounding DINO detector in zero-shot setting, NetTrack reaches 72.5 OWTA versus 70.9 for the best coarse-grained baseline (IOUTracker, line 25) and 69.0 for OC-SORT.
- On TAO open-vocabulary zero-shot, NetTrack gets 32.6 novel TETA, a 4.5-point overall TETA gain over the finetuned baselines per the authors, and an 11.8-point ClsA gain on novel classes.
- On TAO-OW, NetTrack achieves 62.7 known OWTA and 43.7 unknown OWTA in zero-shot, improving known classes over the finetuned SoTA trackers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add fine-grained Net association to Grounding DINO | NetTrack · BFT test | OWTA: 68.7 vs 66.9 | Helped |
| Add fine-grained Nets instead of coarse-grained SORT association (ablation) | Grounding DINO tracking on TAO · TAO novel | TETA: 28.1 -> 32.6 | Helped |
| Add fine-grained Nets instead of coarse-grained ByteTrack association (ablation) | Grounding DINO tracking on TAO · TAO novel | TETA: 32.1 -> 32.6 | Helped |
| Add fine-grained Nets instead of coarse-grained OC-SORT association (ablation) | Grounding DINO tracking on TAO · TAO novel | TETA: 25.7 -> 32.6 | Helped |
| Add fine-grained Nets instead of coarse-grained association (ablation) | Grounding DINO tracking on TAO-OW · TAO-OW unknown | OWTA: 40.9 -> 43.7 | Helped |
| Add fine-grained Nets instead of coarse-grained association (ablation) | Grounding DINO tracking on AnimalTrack · AnimalTrack unknown | OWTA: 48.7 -> 51.4 | Helped |
| Add fine-grained Nets instead of coarse-grained association (ablation) | Grounding DINO tracking on GMOT-40 · GMOT-40 known | OWTA: 44.0 -> 45.9 | Helped |
| Add fine-grained Nets instead of coarse-grained association (ablation) | Grounding DINO tracking on GMOT-40 · GMOT-40 unknown | OWTA: 36.4 -> 36.6 | Helped |
| Replace Grounding DINO with GLIP for localization (ablation) | NetTrack · BFT test | OWTA | No clear effect |
| Replace CoTracker with PIPs for point tracking (ablation) | NetTrack · BFT test | OWTA | No clear effect |
| Replace CoTracker with TAPIR for point tracking (ablation) | NetTrack · BFT test | OWTA | No clear effect |
Surprising findings
- Zero-shot NetTrack outperforms several finetuned closed-set trackers on BFT and TAO-OW, despite no training on the target benchmarks.
- Closed-set trackers lose on average 16% OWTA, 15% HOTA and 21% MOTA when moved from finetuned to zero-shot open-world settings.
Limitations
- Susceptible to many false positives due to absence of finetuning, lowering LocA and AssocA on base classes.
- Pipeline is not end-to-end; the authors suggest a more streamlined end-to-end design as future improvement.
Experimental setup
Datasets: BFT TAO TAO-OW AnimalTrack GMOT-40 TAP-Vid-Kubric
Benchmarks: BFT TAO TAO-OW AnimalTrack GMOT-40
Metrics: OWTA HOTA MOTA IDF1 TETA D. Re. A. Acc.
Baselines: SORT ByteTrack OC-SORT IOUTracker DeepSORT Tracktor++ StrongSORT OVTrack Video OWL-ViT OWTB
Builds on: ByteTrack CoTracker Grounding DINO OC-SORT OWTB
Closest papers by meaning
- VastTrack: Vast Category Visual Object Tracking (NeurIPS 2024 · similarity 0.87)
- Towards Generalizable Multi-Object Tracking (CVPR 2024 · similarity 0.85)
- Omnidirectional Multi-Object Tracking (CVPR 2025 · similarity 0.85)
- Multi-Scene Generalized Trajectory Global Graph Solver with Composite Nodes for Multiple Object Tracking (AAAI 2024 · similarity 0.85)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.85)
- OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.