Research map / Detection and segmentation / Tracking
More Than Meets the Eye: Enhancing Multi-Object Tracking Even with Prolonged Occlusions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MOTE, an end-to-end transformer MOT tracker that adds RAFT optical flow and softmax splatting features (a disocclusion matrix) into track embeddings via an enhanced track embedding module, reporting state-of-the-art results on MOT17, MOT20 and DanceTrack.
Problem
Multi-object tracking under occlusion, especially prolonged occlusions, where appearance-based and frame-by-frame detection methods lose object identity.
Approach
Integrates optical flow with deformable DETR multi-scale features via softmax splatting to form a disocclusion matrix, and fuses this splatted motion-appearance representation into track query embeddings through an enhanced track embedding module (ETEM); the loss is weighted by an IoU-based occlusion mask.
Key results
- MOTE reaches 66.3 HOTA and 82.0 MOTA on MOT17, outperforming StrongSORT's 63.5 HOTA and ByteTrack's 80.3 MOTA.
- On MOT20 MOTE reports 65.8 HOTA and 81.7 MOTA; on DanceTrack 74.2 HOTA and 93.2 MOTA.
- Ablation on MOT17 (5-epoch models) shows softmax splatting beats linear splatting by 3.2 HOTA and 316 fewer identity switches.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MOTE instead of BoostTrack | MOT tracking · MOT17 | HOTA: 65.4 vs. 66.3; 2.8 points | Helped |
| Use MOTE instead of ByteTrack | MOT tracking · MOT17 | MOTA: 1.7 points (82.0 vs. 80.3) | Helped |
| Use MOTE instead of MOTRv2* | MOT tracking · DanceTrack | HOTA: 74.2 vs. 73.4 | Helped |
| Use MOTE instead of MOTR without retraining | MOT tracking · SportsMOT (3 sequences, zero-shot) | MOTA: 45.7% vs. 44.1% | Helped |
| Use softmax splatting rather than linear splatting (ablation) | MOTE · MOT17 (5 epochs) | HOTA: 55.2 -> 58.4; 3.2-point increase | Helped |
| Use softmax splatting rather than linear splatting (ablation) | MOTE · MOT17 (5 epochs) | MOTA: 61.3 -> 64.9; 3.6-point increase | Helped |
| Use 15 optical-flow iterations instead of 20 (ablation) | MOTE · MOT17 | HOTA: 58.3 -> 56.1 | Hurt |
| Use 25 optical-flow iterations instead of 20 (ablation) | MOTE · MOT17 | HOTA: 58.3 -> 57.4 | Hurt |
| Use 25 optical-flow iterations instead of 20 (ablation) | MOTE · MOT17 | MOTA: 63.7 -> 64.5 | Helped |
| Add occlusion weights to the loss (ablation) | MOTE · MOT17 | HOTA: 63.5 -> 66.3; +2.8 | Helped |
| Add occlusion weights to the loss (ablation) | MOTE · MOT17 | MOTA: 80.3 -> 82.0; +1.7 | Helped |
Surprising findings
- The ablation for occlusion weights in Table 9 reports MOTE's MOT17 numbers identical to the main-table numbers, and the 'without occlusion weights' row matches StrongSORT's reported results, raising questions about table consistency.
Limitations
- Struggles with small or distant objects, inherent to optical-flow-based methods.
- Long-term occlusions beyond the current temporal window are not handled.
- Zero-shot SportsMOT evaluation uses only three sequences.
Experimental setup
Datasets: MOT17 MOT20 DanceTrack
Benchmarks: MOT17 MOT20 DanceTrack SportsMOT MOT15
Metrics: HOTA MOTA IDF1 AssA DetA IDS FPS
Baselines: StrongSORT ByteTrack BoostTrack MOTR MOTRv2 OC-SORT QDTrack FairMOT
Builds on: MOTR deformable DETR RAFT softmax splatting MOTRv2
Closest papers by meaning
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.89)
- Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.87)
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking (NeurIPS 2024 · similarity 0.87)
- Sampling-Resilient Multi-Object Tracking (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.