Research map / Detection and segmentation / Tracking
Self-Supervised Multi-Object Tracking with Path Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Path Consistency Loss, a self-supervised objective that enforces consistent object association across observation paths created by randomly skipping frames, enabling long-range matching for multi-object tracking without identity labels.
Problem
Learning robust object matching for multi-object tracking without manual identity annotations, especially over long temporal distances needed to handle occlusion.
Approach
Generate multiple observation paths between the same start and end frames by randomly skipping intermediate frames, and train the matching model so association probability distributions agree across paths (KL to the mean plus entropy minimization), giving supervision over both short and long temporal distances.
Key results
- On MOT17 with private YOLOX detections, achieves 65.0 HOTA and 79.6 IDF1, surpassing the best unsupervised competitor U2MOT (64.2 HOTA, 78.2 IDF1).
- On KITTI car tracking, achieves 78.8 HOTA and 80.3 AssA, exceeding OC-SORT (76.5 HOTA, 76.4 AssA).
- On PersonPath22 with YOLOX detections, achieves 69.4 IDF1 versus ByteTrack's 66.8 IDF1 and reduces ID switches to 4663 vs 5931.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use path consistency loss instead of training with only regularization losses (ablation) | Ours · MOT17 train-set evaluation | IDF1: 3.4 -> 68.9 | Helped |
| Randomly sample start and end frames for paths (ablation) | Path Consistency Loss · MOT17 train-set evaluation | IDF1: 66.6 -> 26.9 | Hurt |
| Remove the spatial constraint mask (ablation) | Path Consistency Loss · MOT17 train-set evaluation | IDF1: 66.6 -> 59.4 | Hurt |
| Remove the one-to-one matching loss (ablation) | Ours · MOT17 train-set evaluation | IDF1: 68.9 -> 62.7 | Hurt |
| Remove the bidirectional consistency loss (ablation) | Ours · MOT17 train-set evaluation | IDF1: 68.9 -> 66.3 | Hurt |
| Limit maximum consecutive skipped frames to s=32 rather than using the full path horizon (ablation) | Path Consistency Loss · Object matching accuracy at temporal distance 32-48 | accuracy: 94.5 -> 93.0 | Hurt |
| Limit maximum consecutive skipped frames to s=24 rather than using the full path horizon (ablation) | Path Consistency Loss · Object matching accuracy at temporal distance 32-48 | accuracy: 94.5 -> 88.0 | Hurt |
| Limit maximum consecutive skipped frames to s=16 rather than using the full path horizon (ablation) | Path Consistency Loss · Object matching accuracy at temporal distance 32-48 | accuracy: 94.5 -> 81.7 | Hurt |
| Limit maximum consecutive skipped frames to s=8 rather than using the full path horizon (ablation) | Path Consistency Loss · Object matching accuracy at temporal distance 32-48 | accuracy: 94.5 -> 71.3 | Hurt |
| Limit maximum consecutive skipped frames to s=4 rather than using the full path horizon (ablation) | Path Consistency Loss · Object matching accuracy at temporal distance 32-48 | accuracy: 94.5 -> 64.1 | Hurt |
Surprising findings
- Models trained only on short-distance matching fail to generalize to long distances, with accuracy at 32-48 frame gaps dropping by about 30 points.
Limitations
- Relies on an off-the-shelf detector for bounding boxes and does not learn detection.
- Frame-pair sampling is a heuristic based on IOU chains, with more advanced selection left to future work.
- Per the paper, some compared methods (e.g., SimpleReID, OUTrack) did not release code, limiting direct comparison.
Experimental setup
Datasets: MOT17 PersonPath22 KITTI
Benchmarks: MOT17 PersonPath22 KITTI
Metrics: HOTA IDF1 MOTA IDsw AssA DetA FP FN
Baselines: SimpleReID U2MOT OUTrack UNS UCSL ByteTrack OC-SORT PointID
Builds on: UNS OUTrack UCSL U2MOT
Closest papers by meaning
- Self-Supervised Multi-Object Tracking with Cross-input Consistency (NeurIPS 2021 · similarity 0.90)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.88)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.87)
- Towards Generalizable Multi-Object Tracking (CVPR 2024 · similarity 0.87)
- ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association (CVPR 2024 · similarity 0.85)
- DeconfuseTrack: Dealing with Confusion for Multi-Object Tracking (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.