Research map / Architectures and efficiency / Attention and transformers
Dynamic Semantic-Aware Correlation Modeling for UAV Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DSATrack, a one-stream ViT UAV tracker with a Dynamic Semantic Relevance Generator that produces semantic-aware correlation maps, plus contribution-ranking layer pruning to yield speed-accuracy variants.
Problem
UAV tracking prioritizes speed over semantic awareness, so search regions fail to extract accurate localization cues from the template under camera motion, fast motion, and low resolution.
Approach
A Dynamic Semantic Relevance Generator builds a learned, prediction-based semantic relevance graph over template tokens (via MLP plus Gumbel-Softmax) that refines the Transformer correlation map, and a hierarchical contribution ranking metric (based on cosine similarity between consecutive layer hidden states) selects Transformer blocks to prune.
Key results
- DSATrack-ViT-B reaches 91.2% precision and 70.6% success on DTB70, and 90.2% precision / 69.4% success on UAV123, the best among compared trackers.
- DSATrack-D7 reaches 162.4 FPS on RTX 3090 and DSATrack-D4 reaches 214.1 FPS, versus 210.7 FPS for sequential pruning of D4.
- On the UAV123 ablation, DSAM improves over baseline Transformer from 87.3/67.2 to 90.2/69.4 precision/success.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamic semantic-aware modeling instead of standard Transformer correlation modeling (ablation) | DSATrack-ViT-B · DTB70 | precision: 87.7 -> 91.2 | Helped |
| add dynamic semantic-aware modeling instead of static semantic-aware modeling (ablation) | DSATrack-ViT-B · UAVDT | precision: 85.5 -> 88.1 | Helped |
| add dynamic semantic-aware modeling (ablation) | DSATrack-ViT-B · UAV123 | precision: 87.3 -> 90.2 | Helped |
| replace standard Transformer correlation modeling with static semantic-aware modeling (ablation) | DSATrack-ViT-B · UAVDT | success: 63.3 -> 62.6 | Hurt |
| rank layer contributions before pruning instead of pruning sequentially (ablation) | DSATrack-D7 · DTB70 | precision: 86.8 -> 88.1 | Helped |
| rank layer contributions before pruning instead of pruning sequentially (ablation) | DSATrack-D4 · UAV123 | success: 60.3 -> 63.0 | Helped |
| add patch embeddings to the template input (ablation) | ViT-D8 · VisDrone2018 | success: 67.4 -> 69.1 | Helped |
| add token elimination (ablation) | ViT-Base · UAV123 | precision: 89.5 -> 90.2 | Helped |
| add token elimination (ablation) | ViT-Base · UAV123 | FPS: 105.6 -> 111.7 | Helped |
| use DSATrack-ViT-B instead of AVTrack-DeiT | UAV tracking · DTB70 | precision: 82.8 -> 91.2 | Helped |
Surprising findings
- The paper reports DSATrack-ViT-B at 111.7 FPS with token elimination but 105.6 FPS without, while the D4 variant reaches 214.1 FPS on RTX 3090 but only about 17 FPS on Xavier.
Limitations
- DSATrack is a local tracker and lacks re-detection after the target is lost.
- Speed on the Jetson AGX Xavier is much lower for larger variants (around 10 FPS for D4/D7 in text, inconsistent with appendix values).
Experimental setup
Datasets: TrackingNet GOT-10k LaSOT COCO
Benchmarks: DTB70 UAVDT VisDrone2018 UAV123 AVisT TrackingNet LaSOT GOT-10k
Metrics: precision success AUC FPS FLOPs Params
Baselines: AVTrack LiteTrack HiT LightFC-ViT SMAT TCTrack TATrack LoReTrack OSTrack AQATrack
Builds on: OSTrack RFGM MAE
Closest papers by meaning
- LVPTrack: High Performance Domain Adaptive UAV Tracking with Label Aligned Visual Prompt Tuning (AAAI 2025 · similarity 0.84)
- JTD-UAV: MLLM-Enhanced Joint Tracking and Description Framework for Anti-UAV Systems (CVPR 2025 · similarity 0.84)
- 3D-aware Select, Expand, and Squeeze Token for Aerial Action Recognition (AAAI 2025 · similarity 0.84)
- Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking (ICML 2024 · similarity 0.84)
- MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking (CVPR 2025 · similarity 0.83)
- Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.