Research map / Detection and segmentation / Tracking
Towards Generalizable Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeneralTrack, a point-wise-to-instance-wise relation tracker using a 4D correlation volume, multi-scale point-region relations and hierarchical part-instance aggregation, aiming for generalization across MOT scenarios without balancing motion and appearance.
Problem
Existing multi-object trackers are tailored to specific scenarios and require hand-tuned balancing of motion and appearance cues, limiting generalization across diverse tracking scenarios and domains.
Approach
Replace instance-level motion/appearance affinity with point-wise dense correlation between consecutive frames, pooled into a multi-scale correlation pyramid acting as an implicit motion template, then aggregated hierarchically (point-part-instance) into tracklet-detection association scores.
Key results
- Ranks 1st on the BDD100K MOT leaderboard with 57.87 mTETA.
- On SportsMOT, achieves 74.1 HOTA-equivalent-level MOTA 96.8 and IDF1 76.4, best IDF1 among compared methods.
- Domain generalization: trained on BDD100K and tested on DanceTrack reaches 54.9 HOTA versus 56.9 when trained in-domain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-scale relation (ablation) | GeneralTrack · BDD100K validation, setting #3 (quadruple downsampling) | HOTA: +2.0 HOTA | Helped |
| add multi-scale relation (ablation) | GeneralTrack · BDD100K validation, setting #3 (quadruple downsampling) | IDF1: +2.7 IDF1 | Helped |
| add multi-scale relation (ablation) | GeneralTrack · BDD100K validation, setting #3 (quadruple downsampling) | MOTA: +2.5 MOTA | Helped |
| change searching radius (ablation) | GeneralTrack · BDD100K validation | HOTA: 63.1 -> 63.5 | No clear effect |
| add hierarchical relation aggregation using flexible-body parts (ablation) | GeneralTrack · BDD100K validation | HOTA | Helped |
| add class relaxation and correction (ablation) | GeneralTrack · BDD100K validation, class average | HOTA: +1.0 | Helped |
| add class relaxation and correction (ablation) | GeneralTrack · BDD100K validation, truck class | IDF1: -1.1 | Hurt |
| train the association model on BDD100K and test without fine-tuning on SportsMOT | GeneralTrack · SportsMOT | HOTA: 73.8 | Mixed |
| train the association model on BDD100K and test without fine-tuning on DanceTrack | GeneralTrack · DanceTrack | HOTA: 54.9 | Mixed |
Surprising findings
- Training only on car-class targets generalizes to all other classes with competitive tracking performance on BDD100K.
Limitations
- Models inter-frame relations only between two consecutive frames; does not extend to cross-frame relations across clips.
- Performance on DanceTrack HOTA is below MOTRv2, which uses far more training resources.
Experimental setup
Datasets: BDD100K SportsMOT DanceTrack MOT17 MOT20
Benchmarks: BDD100K SportsMOT DanceTrack MOT17 MOT20
Metrics: HOTA MOTA IDF1 AssA DetA IDs mHOTA mIDF1 mMOTA TETA
Baselines: ByteTrack GHOST MOTRv2 OC-SORT MixSort MOTR QDTrack Unicorn FineTrack GTR
Builds on: ByteTrack RAFT FairMOT YOLOX F3Net
Closest papers by meaning
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.88)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.87)
- Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- Is Multiple Object Tracking a Matter of Specialization? (NeurIPS 2024 · similarity 0.87)
- Self-Supervised Multi-Object Tracking with Path Consistency (CVPR 2024 · similarity 0.87)
- OVTR: End-to-End Open-Vocabulary Multiple Object Tracking with Transformer (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.