Research map / Detection and segmentation / Tracking
DiffusionTrack: Point Set Diffusion Model for Visual Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Casts visual object tracking as a point-set denoising diffusion process (noise-to-target), refining target point-set hypotheses over multiple diffusion steps per frame to achieve state-of-the-art tracking accuracy in real time.
Problem
Single-forward-pass (one-shot detection) trackers such as Siamese and transformer trackers drift toward similar-looking distractors because they lack self-correction.
Approach
Reformulates tracking as a noise-to-target denoising diffusion process over groups of point sets sampled across the search region, with a decoder of stacked diffusion layers that iteratively refines point-set hypotheses, plus renewal and voting ensemble strategies at inference and early exit for speed.
Key results
- DiffusionTrack-B256 (2 iterations) reaches 75.2% AO on GOT-10k, surpassing OSTrack-256 by 4.2 points and SeqTrack-B256 by 1.0 points.
- DiffusionTrack-L256 achieves 72.3% AUC on LaSOT and 56.8% AUC on TNL2K, the top-ranked results in the comparison table.
- Ablations show Gaussian noise schedule (73.9 AO) beats uniform (72.6) and origin (71.0); removing both ensemble strategies drops multi-iteration AO from 61.1% to 25.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffusionTrack-L256 (1) instead of GRM-256 | visual object tracking · LaSOT | AUC: 69.9 -> 72.3 | Helped |
| use DiffusionTrack-L256 (1) instead of SeqTrack-B256 | visual object tracking · TNL2k | AUC: 54.9 -> 56.8 | Helped |
| use DiffusionTrack-L256 (1) instead of GRM-256 | visual object tracking · TrackingNet | AUC: 84.0 -> 85.2 | Helped |
| use DiffusionTrack-B256 (2) instead of SeqTrack-B256 | visual object tracking · GOT-10k test | AO: 74.7 -> 75.2 | Helped |
| use 2 instead of 5 points per group (ablation) | DiffusionTrack · GOT-10k test | AO: 73.4 -> 74.4 | Helped |
| set the scaling factor to 1.0 instead of 0.1 (ablation) | DiffusionTrack · GOT-10k test | AO: 68.5 -> 74.4 | Helped |
| use a Gaussian instead of Uniform noise schedule (ablation) | DiffusionTrack · GOT-10k test | AO: 72.6 -> 73.9 | Helped |
| use a Gaussian instead of Origin noise schedule (ablation) | DiffusionTrack · GOT-10k test | AO: 71.0 -> 73.9 | Helped |
| increase decoder depth from 1 to 6 layers (ablation) | DiffusionTrack · GOT-10k test | AO: 71.0 -> 74.3 | Helped |
| increase decoder depth from 6 to 12 layers (ablation) | DiffusionTrack · GOT-10k test | AO: 74.3 -> 71.4 | Hurt |
| use early-exit threshold 0.8 instead of 0.6 (ablation) | DiffusionTrack · GOT-10k test | AO: 71.7 -> 73.7 | Helped |
| increase iterative evaluation from 1 to 2 steps (ablation) | DiffusionTrack · GOT-10k test | AO: 74.8 -> 75.2 | Helped |
Surprising findings
- Using a larger signal scaling factor than 1 does not improve results, and too small a scale (0.1) severely hurts AO (74.4 to 68.5), suggesting excessive ground-truth magnitude encourages shortcut learning.
- Increasing point sets per group beyond 2 degrades performance, which the authors attribute to lack of effective supervision for excess points.
Limitations
- Speed-up from early exit is relatively small; the interleaved encoder-decoder variant gains speed but loses notable accuracy.
- Performance gains saturate beyond two evaluation iterations, with suspected overfitting at higher step counts.
Experimental setup
Datasets: COCO LaSOT GOT-10k TrackingNet
Benchmarks: LaSOT TNL2K TrackingNet GOT-10k VOT2022
Metrics: AUC normalized precision precision AO SR0.5 SR0.75
Baselines: OSTrack SeqTrack SimTrack STARK TransT ROMTrack GRM MixFormer SiamRPN++
Builds on: DiffusionDet DDIM RepPoints OSTrack DropTrack pretrained ViT
Closest papers by meaning
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.92)
- DINTR: Tracking via Diffusion-based Interpolation (NeurIPS 2024 · similarity 0.92)
- DeTrack: In-model Latent Denoising Learning for Visual Object Tracking (NeurIPS 2024 · similarity 0.90)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.90)
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.89)
- Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.