Research map / Detection and segmentation / Tracking
MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MUST, the first large-scale multispectral UAV single object tracking dataset (250 sequences, 8 bands), and UNTrack, a one-stream tracker with asymmetric attention, spectral background elimination, and a spectrum prompt encoder.
Problem
RGB UAV trackers struggle with small targets, occlusion, and similar background colors, and no large multispectral UAV tracking dataset existed to study this.
Approach
Unifies spectral, spatial, and temporal features by feeding a spectrum prompt, the initial template, and sequential search frames into one transformer; prunes cross-attention on prompt/template tokens, eliminates background search tokens using spectral-attention confidence, and updates the spectrum prompt from encoded prompt tokens each frame.
Key results
- On MUST, UNTrack reaches 59.7 AUC and 79.2 Precision with reconstructed ImageNet weights, versus 55.1 AUC for OSTrack384*, a 4.6-point gain.
- On HOT, UNTrack achieves 70.4 AUC at 37.0 FPS, the highest AUC and fastest speed among compared trackers.
- On GOT10K (RGB), UNTrack reaches 77.3 AO, comparable to state-of-the-art but not clearly superior (e.g., HIPTrack 77.4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reconstruct the RGB-pretrained parameters for multispectral inputs | UNTrack and comparison trackers · MUST test | AUC: +10% | Helped |
| Use 2 sequential search frames instead of 1 (ablation) | UNTrack · MUST test | AUC: 54.6 -> 59.7 | Helped |
| Increase sequential search frames from 2 to 3 (ablation) | UNTrack · MUST test | AUC: 59.7 -> 59.9 | Helped |
| Increase sequential search frames from 2 to 4 (ablation) | UNTrack · MUST test | AUC: 59.7 -> 58.5 | Hurt |
| Increase sequential search frames from 2 to 5 (ablation) | UNTrack · MUST test | AUC: 59.7 -> 56.3 | Hurt |
| Remove prompt self-attention (ablation) | UNTrack · MUST test | AUC: 59.7 -> 53.8 | Hurt |
| Use full cross-attention for prompt and template instead of pruning it (ablation) | UNTrack · MUST test | AUC: 59.7 -> 55.4 | Hurt |
| Remove spectral background elimination (ablation) | UNTrack · MUST test | AUC: 59.5 -> 59.7 | Hurt |
| Remove the spectrum prompt (ablation) | UNTrack · MUST test | AUC: 59.7 -> 53.9 | Hurt |
| Use a random-initialized prompt without updating (ablation) | UNTrack · MUST test | AUC: 59.7 -> 53.7 | Hurt |
| Generate the prompt from the asymmetric transformer rather than the spectrum prompt encoder (ablation) | UNTrack · MUST test | AUC: 59.7 -> 55.0 | Hurt |
| Replace the spectrum prompt encoder with a naive one-layer convolution encoder (ablation) | UNTrack · MUST test | AUC: 59.7 -> 57.6 | Hurt |
Surprising findings
- Without parameter reconstruction, UNTrack scores only 47.8 AUC on MUST, below the OSTrack384 baseline with reconstructed weights, showing the pretraining mismatch matters greatly.
- Using more than 2 sequential search frames does not consistently improve accuracy, and 3 frames gives only marginal gain while raising compute.
Limitations
- Spectral resolution and frame rate of the dataset are constrained by multispectral imaging cost, limiting high-frame-rate high-band video.
- On RGB GOT10K, UNTrack is only comparable to state-of-the-art, attributed to limited spectral information in RGB images.
Experimental setup
Datasets: MUST HOT GOT10K ImageNet
Benchmarks: MUST HOT GOT10K
Metrics: AUC SR0.5 SR0.75 Precision Normalized Precision AO FPS
Baselines: SiamRPN SiamMask DiMP TaMOs ToMP SGDViT OSTrack ODTrack ZoomTrack MHT BAENet SSTNet
Builds on: OSTrack vision transformer focal loss GIoU loss
Closest papers by meaning
- MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking (NeurIPS 2025 · similarity 0.90)
- UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles (NeurIPS 2024 · similarity 0.85)
- Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking (ICML 2024 · similarity 0.84)
- Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking (CVPR 2025 · similarity 0.84)
- MM-Tracker: Motion Mamba for UAV-platform Multiple Object Tracking (AAAI 2025 · similarity 0.84)
- Dynamic Semantic-Aware Correlation Modeling for UAV Tracking (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.