Research map / Vision-language and multimodal / Robot action and navigation
AURA: Multi-modal Shared Autonomy for Urban Navigation
Yukai Ma, Honglin He, Selina Song, Wayne Wu, Bolei Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune InternVL3-2B on the synthesized MM-CoS instruction labels (ablation) | InternVL3-2B · Instruction following on MM-CoS | ROUGE-L: 0.167 -> 0.532 | Improved |
| finetuned InternVL3-2B versus larger non-finetuned InternVL3-8B (ablation) | InternVL3-2B · Instruction following on MM-CoS | Intent Score: 2.818 -> 4.885 | Improved |
| add SIE together with visual prompts (ablation) | AURA · Instruction following on MM-CoS | ROUGE-L: 0.532 -> 0.581 | Improved |
| add SIE together with visual prompts (ablation) | AURA · Instruction following on MM-CoS | Intent Score: 4.885 -> 5.446 | Improved |
| use SIE alone without visual prompt versus finetuned VLM with visual prompt (ablation) | AURA · Instruction following on MM-CoS | ROUGE-L: 0.532 -> 0.534 | No clear effect |
| remove semantic (visual prompt) encoding from instruction input (ablation) | AURA end-to-end planning · drafting and arrowing planning, 2s L2 | L2 distance at 2s | Hurt |
| add geometry encoding to visual prompts (ablation) | AURA end-to-end planning · long-horizon instruction lag | L2 distance at 2s | Improved |
| add geometry encoding to visual prompts (ablation) | AURA end-to-end planning · short-horizon instruction lag | L2 distance at 2s | No clear effect |
| use arrowing instruction instead of retrained CityWalker point-goal baseline | AURA · MM-CoS open-loop | L2 error at 2s: 0.786 -> 0.473 | Improved |
| use drafting instruction rather than other instruction types | AURA · MM-CoS open-loop | mAP: 0.750 -> 0.844 | Improved |
| give humans high-level instructions instead of point-goal guidance | AURA shared control · pseudo-simulation | Human Operation Frequency: 0.96 -> 0.498 | Improved |
| use AURA instead of baseline navigation models in real-world sidewalk | shared autonomy · Real-world pilot, 2.8 km | Human Operation Ratio (%): 9.74 -> 1.73 | Improved |
Closest papers by meaning
- NaVLA$^2$: A Vision-Language-Audio-Action Model for Multimodal Instruction Navigation (AAAI 2026 · similarity 0.86)
- CitySeeker: How Do VLMs Explore Embodied Urban Navigation with Implicit Human Needs? (ICLR 2026 · similarity 0.86)
- FLAME: Learning to Navigate with Multimodal LLM in Urban Environments (AAAI 2025 · similarity 0.85)
- Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions (NeurIPS 2024 · similarity 0.85)
- Synthesizing Photorealistic and Dynamic Urban Environments for Multimodal Robot Navigation and Collaboration (NeurIPS 2025 · similarity 0.85)
- Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
