Research map / Detection and segmentation / Tracking
DiffusionTrack: Diffusion Model for Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffusionTrack, which casts joint detection and association in multi-object tracking as a denoising diffusion process over paired noise boxes in two adjacent frames, evaluated on MOT17, MOT20, and DanceTrack.
Problem
Multi-object tracking suffers from global/local inconsistency between detection and association, a poor robustness-complexity trade-off, and inflexibility across scenes; existing TBD and JDT methods each have these drawbacks.
Approach
Formulates detection and data association jointly as a consistent noise-to-tracking diffusion process: paired noisy boxes from two adjacent frames are progressively denoised into paired ground-truth boxes, with a spatial-temporal fusion module and association score head in a diffusion head on a frozen YOLOX-backbone feature map, allowing a flexible number of boxes and sampling steps at inference.
Key results
- On MOT17 test, DiffusionTrack reaches 77.9 MOTA and 60.8 HOTA, the best among one-stage trackers, and on MOT20 reaches 72.8 MOTA.
- On DanceTrack test, DiffusionTrack achieves 52.4 HOTA, above OC-SORT's 54.6? no: DiffusionTrack reports the highest HOTA among the listed methods except OC-SORT (54.6), with 89.3 MOTA and 82.2 DetA.
- On MOT17 val-half ablation, a logarithmic perturbation schedule and concatenated Gaussian box padding yield the best results (HOTA 62.6 with log schedule); more boxes or sampling steps improve MOTA at the cost of FPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase prior information proportion from 0% to 50% (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 71.2 -> 74.5 | Helped |
| Increase prior information proportion from 50% to 100% (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 74.5 -> 72.9 | Hurt |
| Replace repeated prior boxes with concatenated Gaussian-noise boxes for padding (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 72.9 -> 73.6 | Helped |
| Use concatenated Poisson-noise boxes instead of concatenated Gaussian-noise boxes (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.6 -> 71.9 | Hurt |
| Use concatenated uniform-noise boxes instead of concatenated Gaussian-noise boxes (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.6 -> 71.5 | Hurt |
| Use concatenated full-image boxes instead of concatenated Gaussian-noise boxes (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.6 -> 71.2 | Hurt |
| Use logarithmic instead of constant perturbation schedule (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.0 -> 74.4 | Helped |
| Use logarithmic instead of linear perturbation schedule (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.6 -> 74.4 | Helped |
| Use logarithmic instead of exponential perturbation schedule (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 74.3 -> 74.4 | Helped |
| Increase sampling steps from 1 to 2 with 500 boxes (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 71.5 -> 71.7 | Helped |
| Increase proposal boxes from 500 to 800 at 1 sampling step (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 71.5 -> 73.6 | Helped |
| Increase proposal boxes from 800 to 1000 at 1 sampling step (ablation) | DiffusionTrack · MOT17 val-half | MOTA: 73.6 -> 74.1 | Helped |
Surprising findings
- Using 100% prior information (repeating previous-frame boxes only) degrades performance relative to 75%, showing that too much prior information hurts.
- The baseline that corrupts only frame t performs very poorly on MOT20 and DanceTrack, indicating that corrupting both frames is necessary.
Limitations
- Performs poorly on small objects in MOT20 due to limitations of the diffusion model.
- Multi-step denoising and long training time are costly, reducing speed.
- Inference relies on a Kalman filter and IoU association for occlusion handling.
Experimental setup
Datasets: MOT17 MOT20 DanceTrack CrowdHuman CityPersons ETHZ
Benchmarks: MOT17 MOT20 DanceTrack
Metrics: MOTA IDF1 HOTA AssA DetA IDs Frag FPS FLOPs
Baselines: ByteTrack OC-SORT BoT-SORT StrongSORT P3AFormer TrackFormer MOTR MeMOT GTR TransTrack CenterTrack QDTrack
Builds on: DiffusionDet YOLOX ByteTrack DDIM GIoU
Closest papers by meaning
- DiffusionTrack: Point Set Diffusion Model for Visual Object Tracking (CVPR 2024 · similarity 0.92)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.91)
- DeTrack: In-model Latent Denoising Learning for Visual Object Tracking (NeurIPS 2024 · similarity 0.91)
- DINTR: Tracking via Diffusion-based Interpolation (NeurIPS 2024 · similarity 0.90)
- DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction (CVPR 2024 · similarity 0.89)
- Multiple Object Tracking as ID Prediction (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.