Research map / Vision-language and multimodal / Robot action and navigation
Reasoning-enhanced vision-language navigation
Adds auxiliary tasks, knowledge, concept learning and structured state modeling to navigation policies.
46 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Robot action and navigation, keyword cluster “language navigation · vln · agent”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
12 papers in 2023–24 and 21 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 15.8% to 4.2%.
Open questions the data points to
- Papers report Baseline on R2R Val Unseen (SPL) anywhere from 42.89 to 52.9: comparisons against it depend on whose number you take.
No longer reported by papers here: R2R Val Unseen (SPL, last in 2024); R2R val unseen (SR, last in 2024); REVERIE Val Unseen (SR, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation (CVPR 2019 · 174 citations)
- 2020 · Vision-Language Navigation With Self-Supervised Auxiliary Reasoning Tasks (CVPR 2020 · 215 citations)
- 2021 · VLN BERT: A Recurrent Vision-and-Language BERT for Navigation (CVPR 2021 · 246 citations)
- 2022 · Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigation (CVPR 2022 · 52 citations)
- 2023 · KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation (CVPR 2023 · 53 citations)
- 2024 · Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation (CVPR 2024 · 28 citations)
- 2025 · Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method (CVPR 2025 · 17 citations)
- 2026 · History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigation (AAAI 2026 · 2 citations)
Earliest papers here
- The Regretful Agent: Heuristic-Aided Navigation Through Progress Estimation (CVPR 2019 · 174 citations)
- Vision-Language Navigation With Self-Supervised Auxiliary Reasoning Tasks (CVPR 2020 · 215 citations)
- Vision-Dialog Navigation by Exploring Cross-Modal Memory (CVPR 2020 · 54 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
R2R Val Unseen · SPL (higher is better)
9 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 50.29 | Vision-Language Navigation With Self-Supervised Auxiliary Reasoning Tasks (CVPR 2020 · 215 citations) |
| 2021 | 61 | History Aware Multimodal Transformer for Vision-and-Language Navigation (NeurIPS 2021 · 21 citations) |
| 2023 | 64 | Frequency-Enhanced Data Augmentation for Vision-and-Language Navigation (NeurIPS 2023 · 10 citations) |
R2R val unseen · SR (higher is better)
9 papers, 2021–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 66 | History Aware Multimodal Transformer for Vision-and-Language Navigation (NeurIPS 2021 · 21 citations) |
| 2023 | 74 | Actional Atomic-Concept Learning for Demystifying Vision-Language Navigation (AAAI 2023 · 3 citations) |
| 2024 | 75 | Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation (ICML 2024 · 0 citations) |
REVERIE Val Unseen · SR (higher is better)
5 papers, 2023–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 50.44 | KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation (CVPR 2023 · 53 citations) |
| 2024 | 57.02 | Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation (ICML 2024 · 0 citations) |
What has been tried
29 papers here report 317 measured changes (a component added, removed or swapped, compared with a baseline): helped 79%, no clear effect 5%, hurt 15%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add region-level semantic alignment | R2R-CE val unseen · SR | 60.20 -> 60.74 | Helped | Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation (CVPR 2024 · 28 citations) |
| add knowledge graph-aware self-attention | REVERIE validation unseen · SR | 45.92 -> 46.18 | Helped | Augmented Commonsense Knowledge for Remote Object Grounding (AAAI 2024 · 18 citations) |
| Add adaptive memory | LHPR-VLN, 3-4 subtasks · ISR | 0. -> 4.69 | Helped | Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method (CVPR 2025 · 17 citations) |
| add auxiliary description D | WebVLN-v1 test · SR | 6.32 -> 12.23 | Helped | WebVLN: Vision-and-Language Navigation on Websites (AAAI 2024 · 11 citations) |
| Replace panoramic action space with egocentric 60° field of view | HA-VLN Validation Seen and Unseen · SR | 0.34 -> 0.20; 0.23 -> 0.16 | Hurt | Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions (NeurIPS 2024 · 4 citations) |
| add backtracking sampling-based data aggregation | Test Seen Full, L1 · SR | 16.10 -> 17.45 | Helped | Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology (ICLR 2025 · 3 citations) |
| add 3D instance and zone representations | R2R-CE Val Unseen · SR | 45.7 -> 52.9 | Helped | Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation (NeurIPS 2025 · 2 citations) |
| add PREVALENT augmentation without full-graph pretraining | Test-R-Basic, Test-N-Basic, Test-N-Scene · SR | 56.8 -> 54.0; 45.4 -> 43.0; 38.2 -> 35.9 | Hurt | General Scene Adaptation for Vision-and-Language Navigation (ICLR 2025 · 0 citations) |
| Add fast update to DUET | REVERIE Val Unseen · SR | 46.98 -> 49.74 | Helped | Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation (ICML 2024 · 0 citations) |
| Add the See2Stop loss during imitation learning | Matterport3D, yoga ball trigger · Att-SR | - -> 75 | Helped | Everyday Object Meets Vision-and-Language Navigation Agent via Backdoor (NeurIPS 2024) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Baseline | R2R Val Unseen | SPL | 42.89 – 52.9 | 2 papers, e.g. CVPR 2020 and AAAI 2022 |
What reviewers will expect
Datasets and baselines named in the 6 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HM3D · 3 of 6 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 33 papers; from OpenAlex and the papers' affiliations).
- Peng Cheng Laboratory · 4 papers
- Chinese Academy of Sciences · 4 papers
- The University of Adelaide · 3 papers
- Sun Yat-sen University · 2 papers
- Peking University · 2 papers
- University of Chinese Academy of Sciences · 2 papers
- National University of Singapore · 2 papers
- Institute of Automation， CAS，China · 2 papers
Use it
Download BibTeX of 35 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Pretraining and data augmentation for VLN (97 papers)
Elsewhere in the topic
- Vision-language-action foundation models (264 papers)
- Memory and subgoal-based manipulation (103 papers)
- Embodied multimodal LLM agents (70 papers)
- Multimodal prompt-driven manipulation (50 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).