Research map / Detection and segmentation / Tracking
SMILEtrack: SiMIlarity LEarning for Occlusion-Aware Multiple Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMILEtrack, a tracking-by-detection tracker that pairs a PRB-Net detector with a Siamese Similarity Learning Module (Patch Self-Attention) and a Similarity Matching Cascade with a GATE function to handle occlusion and similar-looking objects.
Problem
Multiple object tracking under occlusion, visually similar objects, and complex scenes, together with the unclear cost-performance trade-off of tracking-by-detection methods.
Approach
A Siamese appearance similarity module built on patch self-attention (Q-K-V over image patches with CSP-Net backbone) is combined with a two-stage similarity matching cascade in which a GATE function admits high-score unmatched detections as new tracklets only when appearance supports them, rather than relying on a weighted IOU+appearance sum.
Key results
- On MOT17 test, SMILEtrack reaches 81.1 MOTA, 80.5 IDF1 and 65.3 HOTA, versus ByteTrack's 80.3 MOTA and 63.1 HOTA.
- On MOT20 test, SMILEtrack reaches 78.2 MOTA and 63.4 HOTA, versus ByteTrack's 77.8 MOTA and 61.3 HOTA.
- The w/o Re-ID variant runs at 37.5 FPS on MOT17 test while achieving 80.7 MOTA and 65.0 HOTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SLM to ByteTrack with YOLOX (ablation) | ByteTrack · MOT17 val | MOTA: 74.1 -> 76.2 | Helped |
| Add SMC after SLM (ablation) | SMILEtrack with YOLOX · MOT17 val | MOTA: 76.2 -> 76.9 | Helped |
| Add the GATE function after SLM and SMC (ablation) | SMILEtrack with YOLOX · MOT17 val | MOTA: 76.9 -> 77.5 | Helped |
| Add SLM to the PRB-Net detector-only tracker (ablation) | SMILEtrack · MOT17 val | MOTA: 75.3 -> 77.6 | Helped |
| Add SMC after SLM (ablation) | SMILEtrack with PRB-Net · MOT17 val | MOTA: 77.6 -> 78.2 | Helped |
| Add the GATE function after SLM and SMC (ablation) | SMILEtrack with PRB-Net · MOT17 val | MOTA: 78.2 -> 78.6 | Helped |
| Use patch layout E instead of the other tested layouts (ablation) | SLM · MOT17 val | IDF1: 77.4 (A); 77.6 (B); 77.7 (C); 77.9 (D); 78.4 (E); 78.2 (F); 78.4 (G); 78.3 (H) | Helped |
| Use SLM + IOU in Stage I instead of IOU alone (ablation) | SMC · MOT17 val | IDF1: 74.0 -> 78.8 | Helped |
| Use SLM + IOU rather than IOU in Stage II (ablation) | SMC · MOT17 val | MOTA: 76.2 -> 76.1 | No clear effect |
| Use multiple SLM templates with IOU in Stage II (ablation) | SMC · MOT17 val | IDF1: 78.5 -> 78.9 | Helped |
| Add Re-ID (ablation) | SMILEtrack w/o Re-ID · MOT17 test | MOTA: 80.7 -> 81.1 | Helped |
| Replace ByteTrack with SMILEtrack | MOT17 test · MOT17 test | HOTA: 63.1 -> 65.3 | Helped |
Surprising findings
- Adding the SLM, SMC and GATE modules to ByteTrack with YOLOX raises its MOTA to a level comparable to SMILEtrack on the MOT17 validation set.
Limitations
- As an SDE method it is slower than JDE methods, per the authors.
- Full Re-ID variant drops to 5.6 FPS on MOT17 test, far below the w/o Re-ID variant.
- Gains over ByteTrack on MOT17/MOT20 are modest (0.4-0.8 MOTA).
Experimental setup
Datasets: MOT17 MOT20 CalTech PRW CUHK-SYSU COCO
Benchmarks: MOT17 MOT20
Metrics: MOTA IDF1 HOTA FN FP IDs MT ML FPS
Baselines: ByteTrack BoT-SORT StrongSORT++ OC-SORT FairMOT CSTrack TransTrack MAATrack
Builds on: ByteTrack DeepSORT Vision Transformer CSP-Net
Closest papers by meaning
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.88)
- VastTrack: Vast Category Visual Object Tracking (NeurIPS 2024 · similarity 0.87)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.87)
- Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking (AAAI 2024 · similarity 0.86)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.86)
- More Than Meets the Eye: Enhancing Multi-Object Tracking Even with Prolonged Occlusions (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.