Research map / Architectures and efficiency / Attention and transformers
Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ORTrack, a ViT single-stream UAV tracker trained to be invariant to Cox-process random template masking (ORR), plus an adaptive feature-based distillation student (ORTrack-D) for real-time speed.
Problem
Single-stream ViT trackers lack mechanisms to handle target occlusion by buildings and trees in UAV tracking, while real-time efficiency on UAVs is also required.
Approach
Enforce feature invariance of the template under random masking whose locations are sampled from a finite spatial Cox process (bell-shaped intensity) to simulate occlusion, and distill to a smaller student with a loss weighted by the deviation of GIoU loss from its mean (adaptive feature-based KD).
Key results
- ORTrack-DeiT achieves average precision 85.6% and success 65.0% across four UAV benchmarks, the highest among compared trackers, at 226.4 GPU FPS.
- ORR improves ORTrack-DeiT precision on UAVDT from 78.6% to 83.4% (+4.8) and on VisDrone2018 from 81.6 to 88.6 with the Cox masking operator.
- ORTrack-D-DeiT student gives 292.3 GPU FPS and 83.7% average precision, with 1.9% average precision drop versus teacher.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ORR (spatial Cox process masking, MSE invariance loss) (ablation) | ORTrack-ViT · UAVDT | Prec.: +3.3 | Helped |
| add ORR (ablation) | ORTrack-DeiT · UAVDT | Prec.: +4.8 | Helped |
| add ORR (ablation) | ORTrack-Eva · UAVDT | Prec.: +2.7 | Helped |
| add ORR (ablation) | ORTrack-DeiT · UAVDT | Succ.: +3.4 (text says 3.1) | Helped |
| add ORR (ablation) | ARTrack, GRM, DropTrack · UAVDT and VisDrone2018 | Prec. and Succ.: +1.2 to +2.1 Prec. | Helped |
| add AFKD (adaptive feature-based KD) to student (ablation) | ORTrack-DeiT student (ORTrack-D-DeiT) · UAVDT | GPU speed: +36% FPS (ORTrack-DeiT) | Helped |
| add AFKD student distillation (ablation) | ORTrack-DeiT · UAVDT | Prec.: -0.9 (83.4 -> 82.5) | Hurt |
| use spatial Cox process masking (mC) instead of uniform masking (mU) (ablation) | ORTrack-DeiT · VisDrone2018 | Prec.: 88.6 vs 86.7 (mU) | Helped |
| use SAM-based masking instead of spatial Cox masking (ablation) | ORTrack-DeiT · VisDrone2018 | Prec.: 86.8 vs 88.6 | Hurt |
| use CutMix augmentation instead of occlusion masking (ablation) | ORTrack-DeiT · VisDrone2018 | Prec.: 84.3 vs 88.6 | Hurt |
| add adaptive KD loss (AFKD) vs no distillation (ablation) | ORTrack-DeiT student · LaSOT | AUC: +0.9 (53.7 -> 54.6) | Helped |
| add adaptive KD loss (AFKD) vs no distillation (ablation) | ORTrack-DeiT student · TrackingNet | AUC: +0.9 (72.8 -> 73.7) | Helped |
Surprising findings
- SAM-based and CutMix/AdAutoMix masking gave smaller gains than the proposed Cox-process masking on VisDrone2018.
Limitations
- Occlusion robustness is evaluated mainly via a partial-occlusion attribute subset rather than dedicated occlusion benchmarks.
- Real-time claims depend on a single CPU/GPU platform (i9-10850K, TitanX).
Experimental setup
Datasets: GOT-10k LaSOT COCO TrackingNet ImageNet (pretraining)
Benchmarks: DTB70 UAVDT VisDrone2018 UAV123 LaSOT TrackingNet
Metrics: precision success rate AUC normalized precision FPS FLOPs parameters
Baselines: Aba-ViTrack AVTrack-DeiT SMAT TCTrack DRCI AQATrack HIPTrack ROMTrack SeqTrack OSTrack
Builds on: OSTrack MAE Aba-ViTrack feature-based knowledge distillation GIoU loss
Closest papers by meaning
- Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking (ICML 2024 · similarity 0.91)
- Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking (CVPR 2025 · similarity 0.88)
- LVPTrack: High Performance Domain Adaptive UAV Tracking with Label Aligned Visual Prompt Tuning (AAAI 2025 · similarity 0.85)
- MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking (CVPR 2025 · similarity 0.84)
- Open-World Drone Active Tracking with Goal-Centered Rewards (NeurIPS 2025 · similarity 0.83)
- LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.