Research map / Detection and segmentation / Tracking
Exploring Enhanced Contextual Information for Video-Level Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MCITrack, a video-level visual object tracker that stores and transmits rich contextual information in Mamba hidden states, fused into each backbone block via cross-attention.
Problem
Existing video-level trackers transmit contextual information through only a few extra tokens, causing information loss and limiting how much temporal context is captured.
Approach
A Contextual Information Fusion module pairs each backbone block with a Mamba layer whose hidden states record video history, plus in- and out-cross-attention that inject this context into backbone features and write current-frame features back into the state.
Key results
- MCITrack-B224 reaches 75.3% AUC on LaSOT, exceeding ODTrack-B384 (73.2%) with fewer FLOPs (38G vs 73G).
- MCITrack-L384 reaches 76.6% AUC on LaSOT and 80.0% AO on GOT-10k.
- Ablation on LaSOT: Mamba hidden-state propagation (75.3%) beats extra-token (73.4%), previous-features (73.4%) and LSTM (74.3%) propagation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MCITrack-L384 instead of LoRAT-L378 | Large-scale tracker · LaSOT | AUC: 75.1 -> 76.6 | Helped |
| Use MCITrack-L384 instead of LoRAT-L378 | Large-scale tracker · LaSOT_ext | AUC: 56.6 -> 55.7 | Hurt |
| Use MCITrack-L384 instead of ODTrack-L384 | Large-scale tracker · TrackingNet | AUC: 86.1 -> 87.9 | Helped |
| Use MCITrack-L384 instead of ARTrackV2-L384 | Large-scale tracker · GOT-10k | AO: 79.5 -> 80.0 | Helped |
| Use MCITrack-L384 instead of ODTrack-L384 | Large-scale tracker · TNL2K | AUC: 61.7 -> 65.3 | Helped |
| Use MCITrack-L224 instead of ARTrack-L384 | Large-scale tracker · NFS | AUC: 67.9 -> 71.1 | Helped |
| Use MCITrack-L384 instead of LoRAT-L378 | Large-scale tracker · UAV123 | AUC: 72.5 -> 71.5 | Hurt |
| Use 2 CIF blocks instead of 4 (ablation) | MCITrack-B224 · LaSOT | AUC: 75.3 -> 73.9 (-1.4) | Hurt |
| Use 6 CIF blocks instead of 4 (ablation) | MCITrack-B224 · LaSOT | AUC: 75.3 -> 74.8 (-0.5) | Hurt |
| Add in cross-attention (ablation) | MCITrack-B224 CIF block · LaSOT | AUC: 74.3 -> 75.3 (-1.0) | Helped |
| Add out cross-attention (ablation) | MCITrack-B224 CIF block · LaSOT | AUC: 74.2 -> 75.3 (-1.1) | Helped |
| Add both in- and out-cross-attention (ablation) | MCITrack-B224 CIF block · LaSOT | AUC: 73.5 -> 75.3 (-1.8) | Helped |
Surprising findings
- Removing both in- and out-cross-attention causes the largest ablation drop (1.8% AUC), while using 6 frames in the clip performs worse than 4 frames.
Limitations
- Slow training due to video-level modeling.
- Handling video clips adds computational overhead compared to single-frame trackers.
- Longer video clips (6 frames) can degrade performance.
Experimental setup
Datasets: LaSOT GOT-10k TrackingNet COCO VastTrack
Benchmarks: LaSOT LaSOT_ext TrackingNet GOT-10k TNL2K NFS UAV123 VOT2020
Metrics: AUC P_Norm P AO SR0.5 SR0.75 EAO
Baselines: ODTrack-B384 ODTrack-L384 ARTrackV2-256 AQATrack-256 EVPTrack-224 LoRAT-B224 ARTrack-256 SeqTrack-B256 VideoTrack-256 ROMTrack-384 CiteTracker-384 OSTrack-256
Builds on: Mamba OSTrack Fast-iTPN ODTrack LSTM
Closest papers by meaning
- MambaLCT: Boosting Tracking via Long-term Context State Space Model (AAAI 2025 · similarity 0.89)
- Less Is More: Token Context-Aware Learning for Object Tracking (AAAI 2025 · similarity 0.88)
- ODTrack: Online Dense Temporal Token Learning for Visual Tracking (AAAI 2024 · similarity 0.87)
- Robust Tracking via Mamba-based Context-aware Token Learning (AAAI 2025 · similarity 0.86)
- MIMTrack: In-Context Tracking via Masked Image Modeling (AAAI 2025 · similarity 0.85)
- MITracker: Multi-View Integration for Visual Object Tracking (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.