Research map / Vision-language and multimodal / Human and embodied modeling
HIPTrack: Visual Tracking with Historical Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HIPTrack, which adds a lightweight historical prompt network (refined historical foreground masks plus historical visual features in a memory bank) to a frozen Siamese-style one-stream ViT tracker, improving tracking without retraining the backbone.
Problem
Visual object tracking under target appearance variation (deformation, occlusion, scale change) where existing Siamese trackers use only the template or insufficient, expensive historical information, and history-based methods require full retraining with many parameters.
Approach
A historical prompt encoder turns predicted boxes (refined by candidate-elimination masks) and search-region features into key-value memory entries; a decoder uses L2-distance attention between the current search feature and the memory to produce a historical prompt concatenated with search features for the prediction head, with the backbone kept frozen.
Key results
- HIPTrack reaches 72.7 AUC on LaSOT, surpassing ARTrack384 (72.6) and other listed trackers.
- On GOT-10k (trained on train split only), HIPTrack achieves 77.4 AO, 88.0 SR0.5 and 74.5 SR0.75.
- Plugging the historical prompt network into DropTrack raises GOT-10k AO from 75.9 to 77.4; into OSTrack it raises AO from 73.7 to 75.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the historical prompt network to DropTrack | DropTrack · GOT-10k test | AO: 75.9 -> 77.4 | Helped |
| Add the historical prompt network to OSTrack | OSTrack · GOT-10k test | AO: 73.7 -> 75.4 | Helped |
| Add the historical prompt network to SiamFC++ | SiamFC++ · GOT-10k test | AO: 59.5 -> 61.0 | Helped |
| Add historical refined foreground masks (ablation) | HIPTrack · LaSOT test | AUC: 71.5 -> 72.7 | Helped |
| Add historical search-region features (ablation) | HIPTrack · LaSOT test | AUC: 71.2 -> 72.7 | Helped |
| Add candidate-elimination masks to refine foreground masks (ablation) | HIPTrack · LaSOT test | AUC: 72.1 -> 72.7 | Helped |
| Include background-region features in the memory bank (ablation) | HIPTrack · LaSOT test | AUC: 72.3 -> 72.7 | Helped |
| Use L2-distance attention instead of dot-product attention (ablation) | HIPTrack · LaSOT test | AUC: 72.3 -> 72.7 | Helped |
| Vary the number of sampled search frames; 5 gives the highest reported AUC (ablation) | HIPTrack · LaSOT test | AUC: 72.1, 72.5, 72.4, 72.7, 72.4 | Mixed |
| Use HIPTrack instead of ARTrack384 | Visual tracker · TrackingNet test | AUC: 85.1 -> 84.5 | Hurt |
| Use HIPTrack instead of AiATrack | Visual tracker · UAV123 | AUC: 70.6 -> 70.5 | Hurt |
Surprising findings
- Updating the template with an interval as large as 200 frames still outperforms using only the initial template, and ground-truth search-region cropping greatly outperforms predicted-box cropping, indicating large headroom from better history.
Limitations
- Mask-based target position description is less accurate for small targets, as seen on UAV123 where gains are limited.
- Scaling behavior with more sampled search frames is untested; performance peaks at 5 sampled frames.
- Evaluated mainly on Siamese-style one-stream trackers; autoregressive-style integration not explored.
Experimental setup
Datasets: COCO LaSOT GOT-10k TrackingNet
Benchmarks: LaSOT LaSOText GOT-10k TrackingNet UAV123 NfS OTB2015
Metrics: AUC PNorm precision AO SR0.5 SR0.75
Baselines: SeqTrack-B384 ARTrack384 GRM TATrack-B DropTrack OSTrack384 MixFormer-22K STARK AiATrack SwinTrack
Builds on: OSTrack DropTrack STMTrack space-time memory Candidate Elimination
Closest papers by meaning
- STMTrack: Template-Free Visual Tracking With Space-Time Memory Networks (CVPR 2021 · similarity 0.89)
- SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks (CVPR 2019 · similarity 0.88)
- Siamese Instance Search for Tracking (CVPR 2016 · similarity 0.88)
- Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · similarity 0.87)
- DreamTrack: Dreaming the Future for Multimodal Visual Object Tracking (CVPR 2025 · similarity 0.86)
- VastTrack: Vast Category Visual Object Tracking (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.