Research map / Architectures and efficiency / Attention and transformers
Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AVTrack, an adaptive-computation single-stream ViT tracker for UAV tracking that gates transformer blocks with a lightweight Activation Module and learns view-invariant features via MI maximization, plus AVTrack-MD, a multi-teacher MI-based distillation variant with half the blocks.
Problem
Transformer trackers are too slow for real-time UAV tracking on resource-limited devices, and extreme viewpoint changes hurt robustness.
Approach
An Activation Module per ViT block (linear + sigmoid on the first token slice) decides block-level activation to avoid unstructured token-dropping costs; a Deep InfoMax/JSD mutual-information loss between template and interpolated target-patch tokens yields view-invariant features; and a multi-teacher distillation maximizes MI between averaged softened teacher features and student features instead
Key results
- AVTrack-DeiT reaches 84.1% average precision and 64.4% success across five UAV benchmarks at 256.8 GPU FPS on a TitanX.
- AVTrack-MD-DeiT achieves 83.6% precision at 310.6 FPS with 5.3M parameters and 1.5G FLOPs, versus 3.5-7.9M and 0.97-2.4G for AVTrack-DeiT.
- On the averaged five-benchmark comparison, MI-based distillation (JSD) beats MSE distillation by 1.9% precision and 1.7% success for the student.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply MI-based multi-teacher distillation to AVTrack-DeiT | AVTrack-DeiT · Five UAV tracking datasets (Avg.) | GPU FPS: 256.8 -> 310.6 | Helped |
| Apply MI-based multi-teacher distillation to AVTrack-DeiT | AVTrack-DeiT · Five UAV tracking datasets (Avg.) | precision: 84.1 -> 83.6 | Hurt |
| Add AVTrack-ViT and AVTrack-EVA as additional teachers (ablation) | AVTrack-MD-DeiT · Five UAV tracking datasets (Avg.) | precision: 81.7 -> 82.5 -> 83.6 | Helped |
| Increase the student from 4 to 6 ViT blocks (ablation) | AVTrack-MD-DeiT · Five UAV tracking datasets (Avg.) | precision: 81.1 -> 83.6 | Helped |
| Increase the student from 6 to 8 ViT blocks (ablation) | AVTrack-MD-DeiT · Five UAV tracking datasets (Avg.) | GPU FPS: 310.6 -> 272.7 | Hurt |
| Replace MSE distillation loss with MI/JSD loss (ablation) | AVTrack-MD-DeiT student · Five UAV tracking datasets (Avg.) | precision: 81.7 -> 83.6 | Helped |
| Replace MSE distillation loss with MI/JSD loss (ablation) | AVTrack-MD-DeiT student · Five UAV tracking datasets (Avg.) | success rate: 62.4 -> 64.1 | Helped |
Surprising findings
- Aba-ViTrack achieves the highest average precision and success among lightweight trackers, slightly above AVTrack-DeiT (85.3 vs 84.1 precision).
- Some student variants (AVTrack-MD-ViT, AVTrack-MD-EVA) slightly exceed their teacher's average precision despite fewer blocks.
Limitations
- Adaptive block activation yields variable FLOPs and parameter counts, so cost ranges rather than fixed values are reported.
- Some reported gaps to Aba-ViTrack and to larger deep trackers remain in precision and success.
- Speed measured on a single PC (i9-10850K, TitanX) and Jetson AGX, with real-time claims limited to similar or more advanced platforms.
Experimental setup
Datasets: GOT-10k LaSOT COCO TrackingNet
Benchmarks: DTB70 UAVDT VisDrone2018 UAV123 UAV123@10fps WebUAV-3M UAV20L UAVTrack112L
Metrics: precision success rate FPS FLOPs parameters
Baselines: Aba-ViTrack LiteTrack SMAT LightFC SuperSBT PRL-Track HIPTrack EVPTrack ROMTrack
Builds on: OSTrack Deep InfoMax A-ViT DeiT Aba-ViTrack
Closest papers by meaning
- Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking (CVPR 2025 · similarity 0.92)
- Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking (CVPR 2025 · similarity 0.91)
- LVPTrack: High Performance Domain Adaptive UAV Tracking with Label Aligned Visual Prompt Tuning (AAAI 2025 · similarity 0.87)
- Open-World Drone Active Tracking with Goal-Centered Rewards (NeurIPS 2025 · similarity 0.85)
- MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking (CVPR 2025 · similarity 0.84)
- Target Focused Shallow Transformer Framework for Efficient Visual Tracking (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.