Research map / Vision-language and multimodal / Robot action and navigation
Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the LH-VLN task, the NavGen data generation platform, the LHPR-VLN benchmark (3,260 multi-stage tasks), new sub-task metrics (ISR, CSR, CGT), and the MGDM navigation model with CoT feedback and multi-granularity dynamic memory.
Problem
Existing vision-language navigation methods and benchmarks focus on single-stage, short-horizon tasks and do not evaluate long-term planning and decision consistency across multiple consecutive navigation subtasks in complex environments.
Approach
Generates multi-stage navigation tasks with GPT-4 over HM3D scenes in Habitat (forward generation) and decomposes trajectories into step-by-step instructions (backward generation); a navigation model combines CoT feedback with short-term memory blurring/forgetting and long-term memory retrieval.
Key results
- MGDM reaches ISR 4.69, CSR 3.30, CGT 5.83 on 3-4 subtask LH-VLN tasks, while all other baselines score 0 on 2-3 subtasks; fine-tuned NaviLLM gets ISR 3.54 on 3-4 subtasks.
- GPT-4+NaviLLM improves ISR by about 23% over fine-tuned NaviLLM on 3-4 subtask tasks (4.37 vs 3.54).
- Ablation: removing the adaptive memory raises NE from 1.23 to 4.44 and drops ISR/CSR/CGT to 0; removing CoT drops ISR/CSR/CGT to 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MGDM instead of GPT-4 + NaviLLM | LH-VLN navigation · LHPR-VLN, 3-4 subtasks | ISR: 4.37 -> 4.69 | Helped |
| Use MGDM instead of GPT-4 + NaviLLM | LH-VLN navigation · LHPR-VLN, 3-4 subtasks | CSR: 2.91 -> 3.30 | Helped |
| Use MGDM instead of GPT-4 + NaviLLM | LH-VLN navigation · LHPR-VLN, 3-4 subtasks | CGT: 5.23 -> 5.83 | Helped |
| Use MGDM instead of NaviLLM | LH-VLN navigation · Step-by-step LH-VLN | OSR: 6.67 -> 26.92 | Helped |
| Add adaptive memory (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | ISR: 0. -> 4.69 | Helped |
| Add adaptive memory (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | NE: 4.44 -> 1.23 | Helped |
| Add long-term memory (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | ISR: 2.20 -> 4.69 | Helped |
| Add long-term memory (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | NE: 11.13 -> 1.23 | Helped |
| Add CoT feedback (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | ISR: 0. -> 4.69 | Helped |
| Add CoT feedback (ablation) | MGDM · LHPR-VLN, 3-4 subtasks | NE: 2.45 -> 1.23 | Helped |
Surprising findings
- On step-by-step tasks, MGDM has the highest OSR (26.92) and lowest NE (1.70) but SR and SPL are both 0, indicating difficulty in determining goal completion.
- Models perform better on the longer 3-4 subtask tasks than on 2-3 subtask tasks in ISR, CSR and CGT, which the authors attribute to accumulated memory from earlier stages.
Limitations
- All models, including the proposed MGDM, achieve near-zero success rates, so the benchmark remains very hard and results are low.
- MGDM fails to determine when a goal has been reached, yielding zero SR and SPL on step-by-step tasks despite low navigation error.
- Relies on GPT-4 for task generation and CoT feedback, adding cost and dependence on a closed model.
Experimental setup
Datasets: HM3D HSSD LHPR-VLN
Benchmarks: LHPR-VLN
Metrics: SR OSR SPL NE ISR CSR CGT
Baselines: ETPNav GLM-4v prompt NaviLLM GPT-4 + NaviLLM
Builds on: NaviLLM ETPNav Chain-of-Thought prompting Room-to-Room
Closest papers by meaning
- Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-Language Navigation (ICLR 2026 · similarity 0.91)
- DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigation (CVPR 2026 · similarity 0.90)
- Structured Scene Memory for Vision-Language Navigation (CVPR 2021 · similarity 0.90)
- LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation (NeurIPS 2026 · similarity 0.89)
- Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation (AAAI 2026 · similarity 0.88)
- Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.