Research map / Vision-language and multimodal / Robot action and navigation
UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces UAV-Flow, a real-world dataset, simulation suite, and ground-drone deployment framework for language-conditioned fine-grained UAV control via imitation learning, benchmarking VLN and VLA adaptations.
Problem
Language-guided fine-grained, short-range UAV trajectory control (the Flying-on-a-Word task) has been underexplored, as prior work focuses on long-horizon VLN and high-level planning; real-world data for this is lacking.
Approach
Formalizes Flow as mapping state, egocentric observation and atomic language instruction to low-level UAV actions, learned by imitating human pilot trajectories collected in real campuses; adapts VLN (Seq2Seq, CMA, Travel) and VLA (OpenVLA, Pi-0) models to pose regression and deploys them on a real UAV via a ground-drone collaborative look-ahead scheme.
Key results
- VLA models (OpenVLA-UAV, Pi-0-UAV) outperform adapted VLN baselines (Seq2Seq-UAV, CMA-UAV, Travel-UAV) on the UAV-Flow-Sim closed-loop evaluation across 10 Flow task types.
- Seq2Seq-UAV and CMA-UAV perform poorly, drifting and failing to stop at goals due to RNN-based architectures.
- Pi-0-UAV was deployed on a real UAV via the ground-drone collaborative framework with look-ahead, achieving smooth delay-free flight (qualitative only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VLA models instead of VLN models | UAV-Flow-Sim · UAV-Flow-Sim, Flow tasks | success rate | Helped |
| Add the current UAV state as an encoded input (ablation) | Seq2Seq-UAV and CMA-UAV · UAV-Flow | trajectory performance | Hurt |
| Train with the open vocabulary command set (ablation) | OpenVLA-UAV and Pi-0-UAV · UAV-Flow-Sim, Fixed and Open Vocabulary Command Sets | success rate | No clear effect |
| Train with the open vocabulary command set (ablation) | OpenVLA-UAV and Pi-0-UAV · UAV-Flow-Sim, certain tasks | success rate | Helped |
Surprising findings
- Training with the open vocabulary command set did not reduce success rates for OpenVLA-UAV and Pi-0-UAV and sometimes improved certain tasks.
- Pi-0-UAV, strong on object-interactive tasks, performs relatively weaker on fine-grained motion intent instructions.
Limitations
- Real-world evaluation is not quantitative due to flight safety and environmental variability; only qualitative demos.
- Focuses on short-range motion within visual range; no integration with long-horizon planning.
- Simulation discrepancies in visual perception and flight dynamics limit sim-based conclusions.
Experimental setup
Datasets: UAV-Flow UAV-Flow-Sim VLN datasets
Benchmarks: UAV-Flow-Sim
Metrics: success rate NDTW
Baselines: Seq2Seq-UAV CMA-UAV Travel-UAV OpenVLA-UAV Pi-0-UAV
Builds on: OpenVLA Pi-0 Travel Seq2Seq CMA
Closest papers by meaning
- Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology (ICLR 2025 · similarity 0.90)
- WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation (NeurIPS 2026 · similarity 0.88)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.85)
- Open-World Drone Active Tracking with Goal-Centered Rewards (NeurIPS 2025 · similarity 0.85)
- VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actions (NeurIPS 2024 · similarity 0.85)
- Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.