Research map / Reinforcement learning / Imitation learning
Vision-Language Navigation with Energy-Based Policy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Energy-based Navigation Policy (ENP), which models the joint state-action distribution of VLN agents with an energy-based model trained via persistent contrastive divergence and SGLD, and plugs it into existing VLN architectures.
Problem
Vision-language navigation agents trained by behavioural cloning suffer from compounding errors and covariate shift, and fail to match the expert's state-action distribution over full trajectories.
Approach
Reinterpret the VLN action classifier logits as negative energies over state-action pairs, decompose the joint log-likelihood into a cross-entropy action loss plus a marginal state energy term, and estimate its gradient with SGLD sampling initialized from a persistent marginal state memory. The objective is shown to equal forward-KL occupancy-measure matching with the expert.
Key results
- On R2R test unseen, ENP improves VLN-BERT SR from 63 to 65 and DUET SR from 69 to 71.
- On REVERIE test unseen, ENP improves DUET RGS by 1.22 points (from 31.88 to 33.10).
- On RxR-CE val unseen, ENP-ETPNav reaches 55.27% SR and 62.97% NDTW vs 54.79% and 61.90% for ETPNav.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Enp joint state-action policy instead of EnvDrop behavioral cloning | EnvDrop · R2R val unseen | SR: 52 -> 53 | Helped |
| Use Enp joint state-action policy instead of VLN↻BERT | VLN↻BERT · R2R test unseen | SR: 63 -> 65 | Helped |
| Use Enp joint state-action policy instead of DUET | DUET · R2R test unseen | SR: 69 -> 71 | Helped |
| Use Enp joint state-action policy instead of VLN↻BERT | VLN↻BERT · REVERIE val unseen | SR: 35.02 -> 35.80 | Helped |
| Use Enp joint state-action policy instead of ETPNav | ETPNav · RxR-CE val unseen | NDTW: 61.90 -> 62.97 | Helped |
| Add marginal state distribution matching loss to action loss (ablation) | Enp–DUET · R2R val | SR: 72 -> 74 | Helped |
| Add marginal state distribution matching loss to action loss (ablation) | Enp–VLN↻BERT · R2R val | SR: 61 -> 65 | Helped |
| Set SGLD step size ε=1 instead of ε=1.5 (ablation) | Enp–VLN↻BERT · R2R val | SR: 65 -> 64 | Hurt |
| Set SGLD step size ε=2 instead of ε=1.5 (ablation) | Enp–VLN↻BERT · R2R val | SR: 65 -> 64 | Hurt |
| Increase SGLD noise variance from 0.01 to 0.1 (ablation) | Enp–VLN↻BERT · R2R val | SR: 65 -> 63 | Hurt |
| Reduce SGLD iterations from I=15 to I=5 (ablation) | Enp–VLN↻BERT · R2R val | SR: 65 -> 64 | Hurt |
| Increase SGLD iterations from I=15 to I=20 (ablation) | Enp–VLN↻BERT · R2R val | SR: 65 -> 65 | No clear effect |
Surprising findings
- Using only the action loss (degenerating to BC/DAgger) drops DUET R2R val unseen SR from 74 to 72, showing the state marginal term matters.
Limitations
- SGLD inner loop slows training (about 9% delay for 5 iterations).
- Generalization to other VLN architectures and tasks is untested.
- Limited attention to collision avoidance and safety in deployment.
Experimental setup
Datasets: R2R REVERIE R2R-CE RxR-CE
Benchmarks: R2R REVERIE R2R-CE RxR-CE
Metrics: SR SPL NE TL OSR RGS RGSPL NDTW SDTW
Baselines: Seq2Seq SF AuxRN HAMT BEVBert VER BSG LANA HNR AIRL
Builds on: EnvDrop VLN-BERT DUET ETPNav AIRL
Closest papers by meaning
- Active Test-time Vision-Language Navigation (NeurIPS 2025 · similarity 0.88)
- NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning (CVPR 2026 · similarity 0.87)
- Test-Time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning (ICML 2025 · similarity 0.87)
- Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions (NeurIPS 2024 · similarity 0.87)
- GoalLadder: Incremental Goal Discovery with Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.