Research map / Detection and segmentation / Tracking
DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffMOT, a tracking-by-detection tracker whose motion predictor (D2MP) is a one-step decoupled diffusion model conditioned on recent box histories, targeting non-linear motion in multi-object tracking.
Problem
Kalman-filter motion prediction in tracking-by-detection fails on non-linear motion (acceleration, deceleration, irregular direction changes) in scenes like dance and sports, while existing neural motion predictors are either inaccurate or too slow.
Approach
Formulates bounding-box motion prediction as a conditional generative task using a decoupled diffusion process (data-to-zero plus zero-to-noise) with a one-branch network predicting only the constant c, enabling one-step sampling conditioned on the object's previous n=5 frames of box and motion history via an attention-based HMINet.
Key results
- DiffMOT reaches 62.3% HOTA and 63.0% IDF1 on the DanceTrack test set with YOLOX-X at 22.7 FPS, beating Deep OC-SORT by 1.0% HOTA.
- On SportsMOT (detector trained on train+val), DiffMOT achieves 76.2% HOTA, 2.1% above MixSort-OC.
- On DanceTrack validation, D2MP with one-step sampling reaches 55.7% HOTA versus 46.8% for a Kalman Filter, while the two-branch variant needs 20 steps (7.5 FPS) to reach 54.6% HOTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiffMOT instead of Deep OC-SORT | DanceTrack tracker · DanceTrack test | HOTA: 61.3 -> 62.3 | Helped |
| Use DiffMOT instead of MixSort-OC | SportsMOT tracker with train and validation detector · SportsMOT test | HOTA: 74.1 -> 76.2 | Helped |
| Replace D2MP with Kalman Filter (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 46.8 | Hurt |
| Replace D2MP with LSTM (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 51.2 | Hurt |
| Replace D2MP with Transformer (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 54.6 | Hurt |
| Use two-branch D2MP with 1 sampling step instead of one-branch D2MP (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 44.5 | Hurt |
| Use two-branch D2MP with 10 sampling steps (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 52.3 | Hurt |
| Use two-branch D2MP with 20 sampling steps (ablation) | DiffMOT · DanceTrack validation | HOTA: 55.7 -> 54.6 | Hurt |
| Condition on the previous-frame box Bf−1 (ablation) | DiffMOT HMINet · DanceTrack validation | HOTA: 55.7 -> 51.0 | Hurt |
| Condition on the previous-frame motion Mf−1 (ablation) | DiffMOT HMINet · DanceTrack validation | HOTA: 55.7 -> 50.4 | Hurt |
| Set historical motion length to n=3 instead of n=5 (ablation) | DiffMOT HMINet · DanceTrack validation | HOTA: 55.7 -> 53.0 | Hurt |
| Set historical motion length to n=7 instead of n=5 (ablation) | DiffMOT HMINet · DanceTrack validation | HOTA: 55.7 -> 52.5 | Hurt |
Surprising findings
- A two-branch D2MP variant with one-step sampling scores 44.5% HOTA, far below the one-branch version (55.7%), showing that extra parameterization hurts one-step sampling.
- On MOT17 pedestrian scenes DiffMOT (64.5% HOTA) does not beat the best trackers, indicating the method is specialized to non-linear motion.
Limitations
- Does not outperform top trackers on pedestrian-dominant MOT17 (64.5% HOTA vs. 65.1% for SparseTrack).
- Relies on a separate detector (YOLOX) and ByteTrack-style association; gains are tied to the detector quality.
- Motion prediction is applied per object with a fixed history length n=5, and the authors note that too short or too long histories degrade results.
Experimental setup
Datasets: DanceTrack SportsMOT MOT17
Benchmarks: DanceTrack SportsMOT MOT17
Metrics: HOTA AssA DetA IDF1 MOTA FPS
Baselines: Kalman Filter LSTM motion model Transformer motion model IoU only DiffusionTrack OC-SORT Deep OC-SORT MixSort-OC ByteTrack MOTR
Builds on: Decoupled diffusion models (DDM) ByteTrack DDPM Deep OC-SORT
Closest papers by meaning
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.89)
- Sampling-Resilient Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- More Than Meets the Eye: Enhancing Multi-Object Tracking Even with Prolonged Occlusions (ICML 2025 · similarity 0.86)
- S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking (ICML 2025 · similarity 0.86)
- DINTR: Tracking via Diffusion-based Interpolation (NeurIPS 2024 · similarity 0.86)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.