Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Open-World Drone Active Tracking with Goal-Centered Rewards
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DAT, an open-world drone active air-to-ground tracking benchmark with 24 city-scale scenes and digital-twin scene generation, plus GC-VAT, a PPO-based RL tracker using a goal-centered reward and curriculum training.
Problem
Drone visual active tracking with reinforcement learning lacks a unified, realistic benchmark and struggles in open-world environments with interference, where distance-based rewards fail under tilted top-down camera views.
Approach
A goal-centered reward based on a projection-aware deviation metric (instead of Euclidean distance) that gives consistent feedback across unrestricted camera viewpoints, combined with a two-stage curriculum (simple straight-line targets, then complex occluded scenes) trained with PPO.
Key results
- Within-scene TSR of about 0.72 average for GC-VAT versus about 0.19 for D-VAT on the DAT simulator (reported as a 279% relative improvement).
- Cross-scene average TSR of 0.57 for GC-VAT versus 0.19 for D-VAT.
- Zero-shot real-drone deployment on DJI Mini 3 Pro reaches an average TSR of 88.4% and CAR of 81.3%; 81.0% CAR across 24 real-world videos.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the Goal-Centered Reward instead of the distance-based reward (ablation) | GC-VAT · DAT farmland, cross-scene and cross-domain | TSR: about 800% improvement | Helped |
| remove Curriculum-Based Training (ablation) | GC-VAT · DAT farmland, different tests | tracking performance | Hurt |
| add domain randomization (ablation) | GC-VAT · DAT farmland | tracking performance | Helped |
| randomize the drone's initial angle (ablation) | GC-VAT domain randomization · DAT farmland | tracking performance | Helped |
| use GC-VAT instead of D-VAT | DAT · within-scene | CR: 35 -> 242 | Helped |
| use GC-VAT instead of D-VAT | DAT · within-scene | TSR: 0.19 -> 0.72 | Helped |
| use GC-VAT instead of D-VAT | DAT · cross-scene | CR: 37 -> 176 | Helped |
| use GC-VAT instead of D-VAT | DAT · cross-scene | TSR: 0.19 -> 0.57 | Helped |
| use GC-VAT instead of D-VAT | DAT · cross-domain | CR: 35 -> 213 | Helped |
| use GC-VAT instead of D-VAT | DAT · cross-domain | TSR: 0.19 -> 0.67 | Helped |
| add wind perturbations | GC-VAT · DAT citystreet-foggy | TSR: less than 0.06 drop | Hurt |
| add simulated rain | GC-VAT · DAT within-scene, cross-scene, and cross-domain | TSR: less than 0.07 | Hurt |
Surprising findings
- Distance-based rewards from prior work (AOT, D-VAT) learn policies that drive the target out of view, leading to declining reward curves despite low variance.
Limitations
- Real-world evaluation on recorded videos lacks camera control, so only action correctness is measured on real images.
- Real-world drone evaluation is limited to zero-shot deployment with a small-scale setup.
Experimental setup
Datasets: VOT DTB70 UAVDT
Benchmarks: DAT
Metrics: cumulative reward (CR) tracking success rate (TSR) correct action rate (CAR)
Baselines: AOT D-VAT
Builds on: PPO D-VAT AOT webots SUMO
Closest papers by meaning
- WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation (NeurIPS 2026 · similarity 0.85)
- Learning Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking (ICML 2024 · similarity 0.85)
- UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning (NeurIPS 2025 · similarity 0.85)
- Goal-Conditioned On-Policy Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology (ICLR 2025 · similarity 0.84)
- LVPTrack: High Performance Domain Adaptive UAV Tracking with Label Aligned Visual Prompt Tuning (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.