Research map / Detection and segmentation / Tracking
SUTrack: Towards Simple and Unified Single Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SUTrack, a single one-stream transformer tracker trained once on mixed data to handle five SOT tasks (RGB, RGB-Depth, RGB-Thermal, RGB-Event, RGB-Language) via unified 6-channel patch embedding, soft token type embedding, and task-recognition auxiliary loss.
Problem
Single object tracking is fragmented across RGB and multi-modal tasks (depth, thermal, event, language), with separate architectures and models per task, leading to redundant training and limited cross-modal knowledge sharing.
Approach
Unify five SOT tasks in one model and one training session by channel-concatenating RGB with auxiliary-modality images into a 6-channel patch embedding, encoding language with CLIP text features, adding a soft token type embedding that uses bounding-box mask averages, and an auxiliary task-classification loss used only in training.
Key results
- SUTrack-B384 reaches 74.4% AUC on LaSOT, surpassing ODTrack-B384 by 1.2 points.
- SUTrack-L224 achieves 64.0% AUC on VisEvent, surpassing OneTracker by 3.2 points.
- SUTrack-L384 achieves 67.9% AUC on TNL2K, surpassing OneTracker by 9.9 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SUTrack-B384 instead of ODTrack-B384 | RGB tracking · LaSOT | AUC: 73.2 -> 74.4 | Helped |
| use SUTrack-L384 instead of ODTrack-L384 | RGB tracking · TrackingNet | AUC: 86.1 -> 87.7 | Helped |
| use SUTrack-L384 instead of ARTrackV2-L384 | RGB tracking · GOT-10k | AO: 79.5 -> 81.5 | Helped |
| use SUTrack-L384 instead of SDSTrack | RGB-Depth tracking · VOT-RGBD22 | EAO: 72.8 -> 76.6 | Helped |
| use SUTrack-L384 instead of SDSTrack | RGB-Depth tracking · DepthTrack | F-score: 61.9 -> 66.4 | Helped |
| use SUTrack-L384 instead of OneTracker | RGB-Thermal tracking · LasHeR | AUC: 53.8 -> 61.9 | Helped |
| use SUTrack-L224 instead of OneTracker | RGB-Event tracking · VisEvent | AUC: 60.8 -> 64.0 | Helped |
| use SUTrack-L384 instead of OneTracker | RGB-Language tracking · TNL2K | AUC: 58.0 -> 67.9 | Helped |
| train separate single-task models instead of one multi-task model (ablation) | SUTrack-B224 · five SOT task benchmarks | average score change: -7.8 | Hurt |
| exclude the evaluated task's data during training (ablation) | SUTrack-B224 · five SOT task benchmarks | average score change: -11.8 | Hurt |
| add task-recognition auxiliary training (ablation) | SUTrack-B224 · five SOT task benchmarks | average score change: -0.4 | Helped |
| sample RGB data at twice the rate of multimodal data rather than uniformly (ablation) | SUTrack-B224 · five SOT task benchmarks | average score change: -0.9 | Helped |
Surprising findings
- Training separately per task (single-task) underperforms the unified model across all tasks, with a notable -7.8 average drop in the ablation.
Limitations
- Weaker on motion-related attributes such as fast motion, attributed to a small search region and no explicit motion/temporal modeling.
- Multi-modal training data is limited in diversity; uniform sampling of multi-modal data hurts performance.
Experimental setup
Datasets: COCO LaSOT GOT-10k TrackingNet VASTTrack DepthTrack VisEvent LasHeR TNL2K
Benchmarks: LaSOT LaSOT_ext TrackingNet GOT-10k VOT-RGBD22 DepthTrack LasHeR RGBT234 VisEvent TNL2K OTB99 NFS
Metrics: AUC P P_Norm AO SR EAO F-score MSR MPR fps
Baselines: ODTrack LoRAT ARTrackV2 OneTracker SeqTrackv2 SDSTrack Un-Track ViPT ProTrack
Builds on: OSTrack LoRAT HiViT Fast-iTPN CLIP
Closest papers by meaning
- Single-Model and Any-Modality for Video Object Tracking (CVPR 2024 · similarity 0.89)
- VastTrack: Vast Category Visual Object Tracking (NeurIPS 2024 · similarity 0.89)
- UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking (CVPR 2026 · similarity 0.87)
- OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning (CVPR 2024 · similarity 0.87)
- M3SOT: Multi-Frame, Multi-Field, Multi-Space 3D Single Object Tracking (AAAI 2024 · similarity 0.87)
- Towards Category Unification of 3D Single Object Tracking on Point Clouds (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.