Research map / Vision-language and multimodal / Robot action and navigation
Test-Time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FEEDTTA, a test-time adaptation framework for online vision-language navigation that updates a pre-trained VLN policy with REINFORCE on binary episodic success/failure feedback, plus a stochastic gradient reversion regularizer.
Problem
Pre-trained vision-language navigation agents degrade in unseen environments at deployment, and existing test-time adaptation for online VLN (entropy minimization) overfits to failure patterns and limits exploration.
Approach
Adapt the navigation policy online by REINFORCE on a binary success/failure episodic feedback signal (from a human or GPT-4o oracle), and regularize the update with stochastic gradient reversion, which randomly negates a Bernoulli-selected subset of gradient dimensions to balance plasticity and stability.
Key results
- On REVERIE val unseen, FEEDTTA on DUET improves SR from 46.98 to 66.49 (up to 41.53% relative), and FEEDTTA on DUET surpasses state-of-the-art offline methods with a single online stream.
- On R2R val unseen, FEEDTTA on DUET raises SPL by 8.33% relative while reducing NE by 10.88%.
- Using GPT-4o as feedback oracle (72% accuracy) yields DUET test-unseen-adjacent gains; GPT-4o-mini (65% accuracy) also improves over baseline on REVERIE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt DUET with binary episodic feedback and SGR | DUET · REVERIE Val Unseen | SR: 46.98 -> 66.49 | Helped |
| adapt DUET with binary episodic feedback and SGR | DUET · REVERIE Test Unseen | SR: 52.51 -> 53.58 | Helped |
| adapt DUET with binary episodic feedback and SGR | DUET · R2R Val Unseen | SPL: 60 -> 65 | Helped |
| adapt BEVBert with binary episodic feedback and SGR | BEVBert · R2R-CE Val Seen | SR: 68 -> 73 | Helped |
| adapt BEVBert with binary episodic feedback and SGR | BEVBert · R2R-CE Val Unseen | SR: 59 -> 61 | Helped |
| add gradient regularization to FEEDTTA (ablation) | FEEDTTA on DUET · REVERIE Val Seen | SR: 81.80 -> 84.19 | Helped |
| replace gradient dropout with SGR (ablation) | FEEDTTA on DUET · REVERIE Val Seen | SR: 83.06 -> 84.19 | Helped |
| replace positive gradient scaling with SGR (ablation) | FEEDTTA on DUET · REVERIE Val Seen | SR: 84.14 -> 84.19 | No clear effect |
| add gradient regularization to FEEDTTA (ablation) | FEEDTTA on DUET · REVERIE Val Unseen | SR: 63.14 -> 66.49 | Helped |
| replace gradient dropout with SGR (ablation) | FEEDTTA on DUET · REVERIE Val Unseen | SR: 63.36 -> 66.49 | Helped |
| replace positive gradient scaling with SGR (ablation) | FEEDTTA on DUET · REVERIE Val Unseen | SR: 61.63 -> 66.49 | Helped |
| add SGR instead of no regularization (ablation) | FEEDTTA on DUET after adaptation on Val Unseen · REVERIE Val Unseen → Val Seen | SR: 73.72 -> 73.86 | No clear effect |
Surprising findings
- Feedback accuracies below 50% cause clear adaptation failure, while FEEDTTA still outperforms the baseline in SR at 50-60% feedback accuracy.
- A binary episodic goal-based feedback outperformed a step-wise distance-based dense reward (HAMT-style) on REVERIE val unseen (SR 66.49 vs 63.25).
Limitations
- Reliability of LLM oracles (GPT-4o accuracy 72%, GPT-4o-mini 65%) remains a concern.
- Only binary feedback is studied; more advanced feedback systems are left for future work.
- Test-unseen REVERIE results rely on LLM feedback because goal-viewpoint data is unavailable.
Experimental setup
Datasets: REVERIE R2R R2R-CE
Benchmarks: REVERIE R2R R2R-CE
Metrics: SR OSR SPL RGSPL TL NE ASR PSR CSR
Baselines: Tent FSTTA HAMT DUET BEVBert ETPNav VLN⟳BERT HOP+ KERM NaviLLM VLN-VER
Builds on: REINFORCE FSTTA Tent gradient dropout DUET
Closest papers by meaning
- Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation (ICML 2024 · similarity 0.91)
- Efficient Test-Time Adaptation For Robot Policies (NeurIPS 2026 · similarity 0.89)
- Active Test-time Vision-Language Navigation (NeurIPS 2025 · similarity 0.89)
- TTRV: Test-Time Reinforcement Learning for Vision Language Models (CVPR 2026 · similarity 0.88)
- General Scene Adaptation for Vision-and-Language Navigation (ICLR 2025 · similarity 0.88)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.