Research map / Detection and segmentation / Tracking
Efficient Motion Prompt Learning for Robust Visual Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPT, a lightweight plug-and-play motion-prompt module that encodes past bounding-box trajectories and fuses them with frozen vision trackers (OSTrack, SeqTrack, ARTrack) via prompt learning, improving robustness without sequential training.
Problem
Vision-based visual trackers ignore temporal coherence and drift under distractors and occlusion, while existing temporal trackers require costly sequential training.
Approach
A motion encoder with spatial, point, and non-linear temporal positional encodings embeds historical bounding-box trajectories into visual embedding space; a fusion decoder and adaptive weight mechanism combine motion and visual features while the baseline tracker stays frozen, so only the motion modules are trained at frame level.
Key results
- OSTrack-B256 + MPT reaches 48.7 AUC on LaSOT_EXT vs 46.9 for baseline, and SeqTrack-L384 + MPT reaches 73.9 AUC on LaSOT.
- On VOT2018, MPT raises SeqTrack-B256 EAO from 0.339 to 0.428 and OSTrack-B256 EAO from 0.287 to 0.406.
- For OSTrack-B256, MPT trains in about 5 hours with 6G memory versus 38h and 35G for the baseline, adding 13M parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MPT to SeqTrack-L384 | SeqTrack-L384 · LaSOT | AUC: 72.5 -> 73.9 | Helped |
| Add MPT to OSTrack-B384 | OSTrack-B384 · LaSOT_EXT | AUC: 50.9 -> 52.8 | Helped |
| Replace spatial positional encoding with a simple MLP (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 44.8 | Hurt |
| Remove temporal positional encoding (ablation) | OSTrack-B256 + MPT · VOT2022 | EAO: 0.572 -> 0.547 | Hurt |
| Remove point embeddings (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 48.1 | Hurt |
| Use random instead of non-linear TPE initialization (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 45.9 | Hurt |
| Use linear instead of non-linear TPE initialization (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 47.4 | Hurt |
| Disable learning of temporal positional encoding (ablation) | OSTrack-B256 + MPT · VOT2022 | EAO: 0.572 -> 0.552 | Hurt |
| Remove adaptive weight loss L_W (ablation) | OSTrack-B256 + MPT · VOT2022 | EAO: 0.572 -> 0.558 | Hurt |
| Remove motion regression loss L_M and adaptive weight loss L_W (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 45.1 | Hurt |
| Use motion head predictions without fused vision features (ablation) | OSTrack-B256 + MPT · VOT2022 | EAO: 0.572 -> 0.525 | Hurt |
| Remove adaptive weight mechanism (ablation) | OSTrack-B256 + MPT · LaSOT_EXT | AUC: 48.7 -> 46.4 | Hurt |
Surprising findings
- Adding the motion prompt without CutMix augmentation gives slightly worse performance than the vision baseline, per the appendix analysis.
- Failure cases concentrate on trajectories with very low IoU (0.0-0.1), where misleading motion cues hurt tracking.
Limitations
- Tracking accuracy is slightly inferior to sequential-training methods.
- Extremely poor-quality trajectories can mislead the tracker and cause failures.
Experimental setup
Datasets: LaSOT GOT-10K TrackingNet DiMP-18 generated trajectories
Benchmarks: VOT2018 VOT2020 VOT2022 LaSOT LaSOT_EXT TNL2K TrackingNet
Metrics: EAO accuracy robustness AUC normalized precision precision
Baselines: OSTrack SeqTrack ARTrack ROMTrack MixFormer ODTrack ARTrackV2 SiamRCNN TransT STARK
Builds on: OSTrack SeqTrack ARTrack SAM DiMP CutMix
Closest papers by meaning
- MUTrack: A Memory-Aware Unified Representation Framework for Visual Tracking (AAAI 2026 · similarity 0.88)
- Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking (CVPR 2021 · similarity 0.87)
- Explicit Visual Prompts for Visual Object Tracking (AAAI 2024 · similarity 0.87)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.85)
- HIPTrack: Visual Tracking with Historical Prompts (CVPR 2024 · similarity 0.85)
- MemVLT: Vision-Language Tracking with Adaptive Memory-based Prompts (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.