Research map / Detection and segmentation / Tracking
Transformer-based video object tracking
Tracks objects with video transformers, temporal context, point tracking and diffusion features.
74 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Tracking, keyword cluster “visual tracking · visual object · siamese”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 5 of 6 tracked benchmarks.
24 papers in 2023–24 and 32 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 35.8% to 40.0%.
Open questions the data points to
- LaSOT (AUC): best went from 74 (up to 2024) to 76.6 in 2025 — Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations)
- GOT-10k test (AO): best went from 76.6 (up to 2024) to 81.5 in 2025 — Autoregressive Sequential Pretraining for Visual Tracking (CVPR 2025 · 11 citations)
- TrackingNet (AUC): best went from 86.1 (up to 2024) to 87.9 in 2025 — Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations)
- LaSOT_ext (AUC): best went from 53.9 (up to 2024) to 55.7 in 2025 — Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations)
- TNL2K (AUC): best went from 61.7 (up to 2024) to 65.3 in 2025 — Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations)
No longer reported by papers here: LaSOT test (AUC, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Siamese Instance Search for Tracking (CVPR 2016 · 1,156 citations)
- 2018 · A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects (CVPR 2018 · 27 citations)
- 2019 · Graph Convolutional Tracking (CVPR 2019 · 345 citations)
- 2020 · AutoTrack: Towards High-Performance Visual Tracking for UAV With Automatic Spatio-Temporal Regularization (CVPR 2020 · 422 citations)
- 2021 · Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (CVPR 2021 · 866 citations)
- 2022 · TrackFormer: Multi-Object Tracking With Transformers (CVPR 2022 · 1,023 citations)
- 2023 · Unifying Short and Long-Term Tracking With Graph Hierarchies (CVPR 2023 · 97 citations)
- 2024 · ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations)
- 2025 · Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations)
- 2026 · Exploring Reliable Spatiotemporal Dependencies for Efficient Visual Tracking (AAAI 2026 · 2 citations)
Earliest papers here
- Siamese Instance Search for Tracking (CVPR 2016 · 1,156 citations)
- Groupwise Tracking of Crowded Similar-Appearance Targets From Low-Continuity Image Sequences (CVPR 2016 · 26 citations)
- A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects (CVPR 2018 · 27 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
LaSOT · AUC (higher is better)
7 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 74 | ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations) |
| 2025 | 76.6 | Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations) |
GOT-10k test · AO (higher is better)
6 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 68.8 | Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (CVPR 2021 · 866 citations) |
| 2023 | 72.9 | VideoTrack: Learning To Track Objects via Video Transformer (CVPR 2023 · 97 citations) |
| 2024 | 76.6 | Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · 105 citations) |
| 2025 | 81.5 | Autoregressive Sequential Pretraining for Visual Tracking (CVPR 2025 · 11 citations) |
TrackingNet · AUC (higher is better)
6 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 86.1 | ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations) |
| 2025 | 87.9 | Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations) |
LaSOT_ext · AUC (higher is better)
5 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 53.9 | ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations) |
| 2025 | 55.7 | Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations) |
LaSOT test · AUC (higher is better)
4 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 52.1 | GlobalTrack: A Simple and Strong Baseline for Long-Term Tracking (AAAI 2020 · 268 citations) |
| 2023 | 70.2 | VideoTrack: Learning To Track Objects via Video Transformer (CVPR 2023 · 97 citations) |
| 2024 | 72.7 | Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · 105 citations) |
TNL2K · AUC (higher is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 61.7 | ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations) |
| 2025 | 65.3 | Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations) |
What has been tried
57 papers here report 648 measured changes (a component added, removed or swapped, compared with a baseline): helped 72%, no clear effect 4%, hurt 23%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add temporal token propagation to video-level sampling | LaSOT · AUC | 71.0 -> 72.2 | Helped | ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · 200 citations) |
| add temporal decoder and STM | LaSOT test · AUC | 70.5 -> 71.4 | Helped | Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers (CVPR 2024 · 160 citations) |
| replace IoU association with MMD without CMD | MOT17 validation · HOTA | 68.43 -> 68.09 | Hurt | UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation (AAAI 2024 · 115 citations) |
| add the shared embedding rather than feeding mixed modalities directly | DepthTrack test set · F-score | 0.599 -> 0.610 | Helped | Single-Model and Any-Modality for Video Object Tracking (CVPR 2024 · 111 citations) |
| Add spatio-temporal and multi-scale explicit visual prompts | LaSOT · AUC | 69.4 -> 70.4 | Helped | Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · 105 citations) |
| Add modality prompts to a single-branch RGB tracker | LasHeR · Precision | 51.5 -> 65.1 | Helped | Temporal Adaptive RGBT Tracking with Modality Prompt (AAAI 2024 · 83 citations) |
| Use 2 CIF blocks instead of 4 | LaSOT · AUC | 75.3 -> 73.9 (-1.4) | Hurt | Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · 39 citations) |
| Add template updates | LasHeR · AUC | 56.0 -> 57.1 | Helped | Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking (AAAI 2025 · 29 citations) |
| Add dynamic-track filtering to visibility filtering | MPI Sintel · ATE | 0.118 -> 0.085 | Helped | LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry (CVPR 2024 · 27 citations) |
| replace ViT with HiViT | LaSOT · AUC | 69.1% -> 70.5% | Helped | MambaLCT: Boosting Tracking via Long-term Context State Space Model (AAAI 2025 · 25 citations) |
| Replace bidirectional attention with unidirectional attention | GOT-10K test · AO | 73.0 -> 73.9; 0.9% | Helped | Less Is More: Token Context-Aware Learning for Object Tracking (AAAI 2025 · 22 citations) |
| add CKD only | UTB180 · Pre | 62.3 -> 63.6 | Helped | WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark (NeurIPS 2024 · 13 citations) |
What reviewers will expect
Datasets and baselines named in the 17 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- LaSOT · 5 of 17 papers
- GOT-10k · 5 of 17 papers
- TrackingNet · 5 of 17 papers
- COCO · 3 of 17 papers
- VastTrack · 2 of 17 papers
- MOT17 · 2 of 17 papers
- KITTI · 2 of 17 papers
- DAVIS-2017 · 2 of 17 papers
Baselines
- OSTrack · 5 of 17 papers
- SeqTrack · 4 of 17 papers
- STARK · 4 of 17 papers
- ODTrack · 3 of 17 papers
- ARTrackV2 · 3 of 17 papers
- AQATrack · 3 of 17 papers
- MixFormer · 3 of 17 papers
- TAPIR · 3 of 17 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 56 papers; from OpenAlex and the papers' affiliations).
- Guangxi Normal University · 6 papers
- Korea Advanced Institute of Science and Technology · 4 papers
- Tsinghua University · 3 papers
- Harbin Institute of Technology · 3 papers
- University of Chinese Academy of Sciences · 3 papers
- Shanghai Jiao Tong University · 3 papers
- Columbia University · 2 papers
- Wuzhou University · 2 papers
- Zhejiang University · 2 papers
- Xidian University · 2 papers
Use it
Download BibTeX of 34 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Foundation-model and correlation-filter tracking (66 papers)
- Siamese network tracking (34 papers)
Elsewhere in the topic
- Association cues for online MOT (60 papers)
- Memory and correlation-based MOT (52 papers)
- Unified transformer tracking (25 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).