Research map / Vision-language and multimodal / Robot action and navigation
A New Path: Scaling Vision-and-Language Navigation With Synthetic Instructions and Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Speaker-Matterport synthetic instructions with higher-quality Marky-Matterport instructions (ablation) | MARVAL pretraining · RxR Val-Unseen | SR: +27% SR on RxR | Helped |
| Add Speaker-Matterport instructions alongside Marky-Matterport instructions (ablation) | MARVAL pretraining · RxR and R2R Val-Unseen | NDTW: 60.3 -> 56.9 | Hurt |
| Add Gibson environments with Marky-generated instructions (ablation) | MARVAL pretraining · RxR Val-Unseen | SR: +11% SR on RxR | Helped |
| Add SE3DS novel-view image synthesis without Gibson data (ablation) | MARVAL pretraining · RxR Val-Unseen | SR: +6% SR on RxR | Helped |
| Add SE3DS novel-view image synthesis when Gibson data is already included (ablation) | MARVAL pretraining · RxR Val-Unseen | SR: +0% SR on RxR | No clear effect |
| Scale the transformer from mT5-base to mT5-large (ablation) | MARVAL pretraining · RxR Val-Unseen | SR: +2% SR on RxR | Helped |
| Finetune the pretrained agent on human annotations using behavioral cloning (ablation) | MARVAL · RxR Val-Unseen | NDTW: 67.0 -> 70.6 | Helped |
| Add DAGGER finetuning after behavioral cloning (ablation) | MARVAL · RxR Val-Unseen | NDTW: 70.6 -> 70.8 | Helped |
| Self-train with Marky-generated instructions in the evaluation environments (ablation) | MARVAL · RxR Test (Unseen) | NDTW: 66.8 -> 68.6 | Helped |
| Use MARVAL trained on RxR-domain synthetic instructions on the R2R benchmark | MARVAL · R2R Val-Unseen | SR: 64.8 vs. 68.9 | Hurt |
Closest papers by meaning
- Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-Training (CVPR 2020 · similarity 0.92)
- Iterative Vision-and-Language Navigation (CVPR 2023 · similarity 0.91)
- General Scene Adaptation for Vision-and-Language Navigation (ICLR 2025 · similarity 0.91)
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · similarity 0.90)
- NavBench: Probing Multimodal Large Language Models for Embodied Navigation (NeurIPS 2025 · similarity 0.90)
- Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.