Research map / Large language models / Reasoning and chain of thought
ARTrackV2: Prompting Autoregressive Tracker Where to Look and How to Describe
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ARTrackV2, an encoder-only generative tracker that jointly autoregresses trajectory tokens and appearance tokens (reconstructed feature maps) across frames, dropping intra-frame decoding for ~3.6x speedup.
Problem
Visual object tracking needs both accurate localization (where to look) and appearance adaptation (how to describe the target); prior generative trackers either model only trajectory with slow intra-frame autoregressive decoding or rely on hand-tuned, multi-stage appearance update rules.
Approach
Jointly model trajectory and appearance evolution in a time-autoregressive generative framework using a pure transformer encoder: appearance tokens act as dynamic templates reconstructed each frame (feature-domain MSE), guided by a learned IoU confidence token, with attention masking, sequence-level reverse augmentation, and no intra-frame decoding.
Key results
- ARTrackV2-L_384 reaches 79.5% AO on GOT-10k (one-shot), exceeding ARTrack-L_384's 78.5%.
- ARTrackV2-L_384 achieves 86.1% AUC on TrackingNet and 73.6% AUC on LaSOT.
- ARTrackV2_256 runs at 94 FPS versus ARTrack's 26 FPS (about 3.6x faster), with 75.9% AO on GOT-10k.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add appearance evolution to the pure encoder (ablation) | ARTrackV2 · GOT-10k test | AO: 71.0 -> 74.2 | Helped |
| add confidence prediction (ablation) | ARTrackV2 · GOT-10k test | AO: 74.2 -> 74.7 | Helped |
| add the masking strategy (ablation) | ARTrackV2 · GOT-10k test | AO: 74.7 -> 75.2 | Helped |
| add sequence augmentation (ablation) | ARTrackV2 · GOT-10k test | AO: 75.2 -> 75.9 | Helped |
| use feature reconstruction instead of image reconstruction (ablation) | ARTrackV2 appearance model · GOT-10k test | AO: 74.8 -> 75.9 | Helped |
| use IoU as the appearance evolution indicator instead of the alternatives (ablation) | ARTrackV2 appearance model · GOT-10k test | AO: 75.1 -> 75.9 | Helped |
| use reverse-video augmentation instead of fixed or random interval sampling (ablation) | ARTrackV2 · GOT-10k test | AO: +0.7% | Helped |
| use fixed-interval or random-interval sampling instead of reverse augmentation (ablation) | ARTrackV2 · GOT-10k test | precision: null | Hurt |
| remove trajectory evolution (ablation) | ARTrackV2-256 · GOT-10k test | AO: 75.9 -> 72.9 | Hurt |
| remove appearance evolution (ablation) | ARTrackV2-256 · GOT-10k test | AO: 75.9 -> 72.3 | Hurt |
| remove both trajectory and appearance evolution (ablation) | ARTrackV2-256 · GOT-10k test | AO: 75.9 -> 69.8 | Hurt |
| increase appearance-token masking ratio to 90% (ablation) | ARTrackV2 · GOT-10k test | AO: 74.7, 75.0, 75.2, 75.5, 75.9 | Helped |
Surprising findings
- Masking appearance tokens improves accuracy up to a 90% masking ratio, unlike BERT and MAE where accuracy peaks at lower ratios.
- Image-pixel reconstruction of the appearance underperforms feature-domain reconstruction (74.8 vs 75.9 AO).
Limitations
- Pure encoder variant alone loses accuracy (71.0 AO), requiring the appearance and augmentation components to recover it.
- Visibility-based indicator performs poorly due to noisy GOT-10k visibility labels.
- Training requires long runs (26 to 120 hours on 8 A6000 GPUs) and a 32-frame sequence limit set by GPU memory.
Experimental setup
Datasets: GOT-10k TrackingNet LaSOT COCO2017
Benchmarks: GOT-10k TrackingNet LaSOT LaSOT_ext TNL2K NFS UAV123
Metrics: AO SR0.5 SR0.75 AUC P_Norm P FPS
Baselines: ARTrack SeqTrack OSTrack STARK MixViT GRM TATrack CTTrack
Builds on: ARTrack SeqTrack MixFormer MAE
Closest papers by meaning
- Autoregressive Sequential Pretraining for Visual Tracking (CVPR 2025 · similarity 0.80)
- Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · similarity 0.79)
- Autoregressive Queries for Adaptive Tracking with Spatio-Temporal Transformers (CVPR 2024 · similarity 0.79)
- HIPTrack: Visual Tracking with Historical Prompts (CVPR 2024 · similarity 0.77)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.77)
- Dynamic Updates for Language Adaptation in Visual-Language Tracking (CVPR 2025 · similarity 0.76)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.