Research map / Architectures and efficiency / Attention and transformers
Robust Tracking via Mamba-based Context-aware Token Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TemTrack, a one-stream visual tracker that adds one learnable track token per frame and models temporal context over a sliding window of these tokens with a Mamba+cross-attention Temporal Module, instead of feeding more template images.
Problem
Visual object tracking needs good performance at low computational cost, but existing temporal trackers feed multiple extra images or features, increasing compute and learning burden and introducing interfering information.
Approach
Represent each frame's target appearance as a single track token inside the backbone, and learn temporal relations by interacting these tokens within a sliding window using a Mamba layer followed by cross-attention; the resulting token then guides the search features before the head.
Key results
- TemTrack-256 reaches 72.0% AUC on LaSOT, outperforming AQATrack-256 (71.4%) by 0.6 points.
- TemTrack-384 reaches 76.1% AO on GOT-10k and 73.1% AUC on LaSOT, running at 36 fps on a V100 with 55.7G FLOPs.
- Ablation on LaSOT shows the Mamba_Cross Temporal Module adds +0.9 AUC over the track-token-only variant (71.4 to 72.0 with Fast-iTPN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add track token (ablation) | TemTrack with Fast-iTPN · LaSOT | AUC: 71.1 -> 71.4 | Helped |
| add track token (ablation) | TemTrack with Fast-iTPN · GOT-10k | AO: 73.0 -> 73.7 | Helped |
| add Temporal Module after adding the track token (ablation) | TemTrack with Fast-iTPN · LaSOT | AUC: 71.4 -> 72.0 | Helped |
| add Temporal Module after adding the track token (ablation) | TemTrack with Fast-iTPN · GOT-10k | AO: 73.7 -> 74.9 | Helped |
| replace Mamba with self-attention in the first temporal layer (ablation) | TemTrack Temporal Module · GOT-10k | AO: 74.9 -> 74.3 | Hurt |
| use self-attention in both temporal layers instead of Mamba followed by cross-attention (ablation) | TemTrack Temporal Module · GOT-10k | AO: 74.9 -> 74.2 | Hurt |
| add proposed method with ViT-B backbone (ablation) | ViT-B tracker · LaSOT | AUC: +1.0 | Helped |
| add proposed method with HiViT backbone (ablation) | HiViT tracker · LaSOT | AUC: +0.6 | Helped |
| add proposed method with Fast-iTPN backbone (ablation) | Fast-iTPN tracker · LaSOT | AUC: +0.9 | Helped |
| increase sliding-window size from m=2 to m=4 (ablation) | TemTrack · LaSOT | AUC: 71.1 -> 71.9 | Helped |
| increase sliding-window size from m=4 to m=8 (ablation) | TemTrack · LaSOT | AUC: 71.9 -> 72.0 | Helped |
Surprising findings
- Self-attention-only Temporal Module variants (Self_Cross, Self_Self) still perform close to the Mamba variant on GOT-10k (74.3 and 74.2 AO vs 74.9).
Limitations
- Gains over prior trackers are small (e.g., 0.6 AUC on LaSOT over AQATrack-256).
- Best sliding-window size only explored at a few values (m = 2, 4, 8), with the optimum described as between 4 and 8.
Experimental setup
Datasets: COCO LaSOT TrackingNet GOT-10k
Benchmarks: LaSOT LaSOT_ext GOT-10k TrackingNet UAV123 TNL2K
Metrics: AUC P_norm precision AO SR0.5 SR0.75 FLOPs Params FPS
Baselines: AQATrack EVPTrack F-BDMTrack ROMTrack ARTrack SeqTrack VideoTrack MixFormer OSTrack STARK HIPTrack ODTrack
Builds on: OSTrack Fast-iTPN STARK SeqTrack
Closest papers by meaning
- MambaLCT: Boosting Tracking via Long-term Context State Space Model (AAAI 2025 · similarity 0.90)
- Less Is More: Token Context-Aware Learning for Object Tracking (AAAI 2025 · similarity 0.88)
- ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · similarity 0.87)
- Exploring Enhanced Contextual Information for Video-Level Object Tracking (AAAI 2025 · similarity 0.86)
- Foundation Model Driven Appearance Extraction for Robust Multiple Object Tracking (AAAI 2025 · similarity 0.85)
- MIMTrack: In-Context Tracking via Masked Image Modeling (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.