Research map / Detection and segmentation / Tracking
Foundation-model and correlation-filter tracking
Unify visual tracking with foundation models, diffusion, and overlap or correlation-filter trackers.
66 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Tracking, keyword cluster “visual tracking · visual object · siamese”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 3 of 6 tracked benchmarks.
10 papers in 2023–24 and 6 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 14.9% to 7.5%.
Open questions the data points to
- LaSOT (AUC): best went from 72.9 (up to 2024) to 75.1 in 2025 — A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations)
- GOT-10k test (AO): best went from 77.9 (up to 2024) to 79.9 in 2025 — DreamTrack: Dreaming the Future for Multimodal Visual Object Tracking (CVPR 2025 · 11 citations)
- GOT-10k (AO): best went from 74.3 (up to 2024) to 81.1 in 2025 — A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations)
No longer reported by papers here: VOT2016 (EAO, last in 2020); VOT2018 (EAO, last in 2021); VOT2018 (Accuracy, last in 2020).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Staple: Complementary Learners for Real-Time Tracking (CVPR 2016 · 1,851 citations)
- 2017 · Discriminative Correlation Filter With Channel and Spatial Reliability (CVPR 2017 · 1,189 citations)
- 2018 · Fast and Accurate Online Video Object Segmentation via Tracking Parts (CVPR 2018 · 302 citations)
- 2019 · LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking (CVPR 2019 · 1,706 citations)
- 2020 · D3S - A Discriminative Single Shot Segmentation Tracker (CVPR 2020 · 312 citations)
- 2021 · Towards More Flexible and Accurate Object Tracking With Natural Language: Algorithms and Benchmark (CVPR 2021 · 281 citations)
- 2022 · Transformer Tracking With Cyclic Shifting Window Attention (CVPR 2022 · 237 citations)
- 2023 · Representation Learning for Visual Object Tracking by Masked Appearance Transfer (CVPR 2023 · 78 citations)
- 2024 · OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning (CVPR 2024 · 139 citations)
- 2025 · A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations)
- 2026 · RELO: Reinforcement Learning to Localize for Visual Object Tracking (ICML 2026 · 0 citations)
Earliest papers here
- Staple: Complementary Learners for Real-Time Tracking (CVPR 2016 · 1,851 citations)
- Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking (CVPR 2016 · 421 citations)
- Beyond Local Search: Tracking Objects Everywhere With Instance-Specific Proposals (CVPR 2016 · 200 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
LaSOT · AUC (higher is better)
9 papers, 2020–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 48.1 | POST: POlicy-Based Switch Tracking (AAAI 2020 · 17 citations) |
| 2022 | 66.2 | Transformer Tracking With Cyclic Shifting Window Attention (CVPR 2022 · 237 citations) |
| 2023 | 70.3 | Reading Relevant Feature from Global Representation Memory for Visual Object Tracking (NeurIPS 2023 · 5 citations) |
| 2024 | 72.9 | DeTrack: In-model Latent Denoising Learning for Visual Object Tracking (NeurIPS 2024 · 1 citation) |
| 2025 | 75.1 | A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations) |
VOT2016 · EAO (higher is better)
6 papers, 2017–2020 · not reported since 2020
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 0.338 | Discriminative Correlation Filter With Channel and Spatial Reliability (CVPR 2017 · 1,189 citations) |
| 2019 | 0.391 | Visual Tracking via Adaptive Spatially-Regularized Correlation Filters (CVPR 2019 · 455 citations) |
| 2020 | 0.493 | D3S - A Discriminative Single Shot Segmentation Tracker (CVPR 2020 · 312 citations) |
VOT2018 · EAO (higher is better)
6 papers, 2019–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 0.401 | ATOM: Accurate Tracking by Overlap Maximization (CVPR 2019 · 1,583 citations) |
| 2020 | 0.489 | D3S - A Discriminative Single Shot Segmentation Tracker (CVPR 2020 · 312 citations) |
GOT-10k test · AO (higher is better)
5 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 70.1 | Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation (CVPR 2021 · 231 citations) |
| 2024 | 77.9 | DeTrack: In-model Latent Denoising Learning for Visual Object Tracking (NeurIPS 2024 · 1 citation) |
| 2025 | 79.9 | DreamTrack: Dreaming the Future for Multimodal Visual Object Tracking (CVPR 2025 · 11 citations) |
VOT2018 · Accuracy (higher is better)
5 papers, 2019–2020 · not reported since 2020
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 0.59 | ATOM: Accurate Tracking by Overlap Maximization (CVPR 2019 · 1,583 citations) |
| 2020 | 0.64 | D3S - A Discriminative Single Shot Segmentation Tracker (CVPR 2020 · 312 citations) |
GOT-10k · AO (higher is better)
4 papers, 2022–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 69.4 | Transformer Tracking With Cyclic Shifting Window Attention (CVPR 2022 · 237 citations) |
| 2023 | 74.1 | Reading Relevant Feature from Global Representation Memory for Visual Object Tracking (NeurIPS 2023 · 5 citations) |
| 2024 | 74.3 | Sequential Fusion Based Multi-Granularity Consistency for Space-Time Transformer Tracking (AAAI 2024 · 14 citations) |
| 2025 | 81.1 | A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations) |
What has been tried
64 papers here report 643 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 4%, hurt 21%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove CMT Prompters | DepthTrack · F | 56.5 -> 60.9 | Hurt | OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning (CVPR 2024 · 139 citations) |
| add similarity-matrix distillation (SKD) | COESOT · PR/SR | 61.8/50.9 -> 62.5/51.6 | Helped | Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline (CVPR 2024 · 69 citations) |
| use 2 instead of 5 points per group | GOT-10k test · AO | 73.4 -> 74.4 | Helped | DiffusionTrack: Point Set Diffusion Model for Visual Object Tracking (CVPR 2024 · 43 citations) |
| Update memory only when the predicted mask is non-empty | DiDi · Quality | 0.649 -> 0.665 | Helped | A Distractor-Aware Memory for Visual Object Tracking with SAM2 (CVPR 2025 · 42 citations) |
| add LCL | LaSOT · AUC | 71.1 -> 71.4 | Helped | Sequential Fusion Based Multi-Granularity Consistency for Space-Time Transformer Tracking (AAAI 2024 · 14 citations) |
| add Future Dreaming Module | LaSOT · SUC | 69.3 -> 71.7 | Helped | DreamTrack: Dreaming the Future for Multimodal Visual Object Tracking (CVPR 2025 · 11 citations) |
| use a single forward pass instead of multiple forward passes | GOT-10k test · AO | 75.9 -> 77.1 | Helped | DeTrack: In-model Latent Denoising Learning for Visual Object Tracking (NeurIPS 2024 · 1 citation) |
| Add streaming module | TAP-Vid DAVIS · AJ | 64.2 -> 65.1; 0.9 AJ | Helped | Lattice Boltzmann Model for Learning Real-World Pixel Dynamicity (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 10 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- LaSOT · 6 of 10 papers
- GOT-10k · 6 of 10 papers
- TrackingNet · 5 of 10 papers
- COCO · 4 of 10 papers
- VisEvent · 2 of 10 papers
Baselines
- MixFormer · 7 of 10 papers
- STARK · 7 of 10 papers
- TransT · 7 of 10 papers
- OSTrack · 5 of 10 papers
- SeqTrack · 4 of 10 papers
- GRM · 3 of 10 papers
- ROMTrack · 3 of 10 papers
- AiATrack · 2 of 10 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 16 papers; from OpenAlex and the papers' affiliations).
- Fudan University · 3 papers
- Shanghai Jiao Tong University · 2 papers
- University of Chinese Academy of Sciences · 2 papers
Use it
Download BibTeX of 32 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Siamese network tracking (34 papers)
- Transformer-based video object tracking (74 papers)
Elsewhere in the topic
- Association cues for online MOT (60 papers)
- Memory and correlation-based MOT (52 papers)
- Unified transformer tracking (25 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).