Research map / Vision-language and multimodal / Robot action and navigation
WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WorldVLN, an autoregressive world-action model that adapts a latent video backbone (InfinityStar) to predict short-horizon latent transitions decoded into waypoint actions for aerial VLN, trained with SFT then Action-aware GRPO.
Problem
Aerial vision-language navigation requires closed-loop action generation, but VLA models lack modeling of how the world evolves under the agent's actions, and bidirectional video world models are mismatched to causal observe-act-update loops.
Approach
Repurpose a pretrained latent autoregressive video transformer to predict short-horizon latent world-state transitions, decode them directly into waypoint actions via an action decoder, feed real observation latents back into context, and optimize segment-level waypoint decisions with GRPO using trajectory, task-progress, and reference rewards with temporal decay.
Key results
- WorldVLN reaches 79.12% average SR (Fixed) and 78.02% (Open) on UAV-Flow-Sim, outperforming the strongest baseline pi0-UAV by about 13.5 and 12.2 points.
- On IndoorUAV-VLA, WorldVLN achieves 41.76% full-set SR and 13.48% NDTW, versus 27.16% SR for pi0 (+14.60 points).
- Autoregressive SFT improves success over full-sequence SFT by 5.7+ points on UAV-Flow and IndoorUAV according to the paper's ablation; Action-aware GRPO adds over 10 points after SFT saturates.
Surprising findings
- Full-sequence latent prediction shows semantic drift and scene collapse, whereas autoregressive updating keeps latent representations coherent.
Limitations
- Validated only on short-range aerial navigation and short-horizon waypoint generation.
- Real-world deployment requires server-side inference and cannot run fully onboard.
- No multi-seed experiments conducted, limiting statistical validation.
Experimental setup
Datasets: UAV-Flow IndoorUAV-VLA
Benchmarks: UAV-Flow-Sim IndoorUAV-VLA
Metrics: success rate NDTW
Baselines: Seq2Seq-UAV CMA-UAV Travel-UAV OpenVLA-UAV pi0-UAV GPT-4o Seq2Seq CMA OpenVLA pi0-FAST NaVid pi0
Builds on: InfinityStar GRPO OpenVLA Wan VAE
Closest papers by meaning
- AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild (ICLR 2026 · similarity 0.92)
- Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology (ICLR 2025 · similarity 0.91)
- World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.90)
- SatNav: A Scalable Benchmark for Long-Horizon UAV Vision-Language Navigation from Satellite Imagery (NeurIPS 2026 · similarity 0.89)
- UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning (NeurIPS 2025 · similarity 0.88)
- RLVR-World: Training World Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.