Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Active Test-time Vision-Language Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ATENA, a test-time active learning framework for vision-language navigation that uses episodic success/failure feedback (human or self-predicted) with mixture entropy optimization to adapt offline-trained VLN policies.
Problem
Offline-trained VLN policies degrade under distribution shift at test time, and standard entropy minimization can amplify overconfident errors in sequential navigation.
Approach
Mixture entropy optimization minimizes entropy of a mix of the action distribution and a one-hot pseudo-expert on the chosen action for successful episodes and maximizes it for failed ones; a self-prediction head trained online supplies success labels for confident episodes, while human episodic feedback is requested for uncertain ones.
Key results
- On REVERIE val unseen, ATENA with DUET improves SR from 46.98 to 68.11 (about +21 points) and with GOAT from 53.82 to 67.66.
- On R2R val unseen, ATENA with BEVBert raises SR from 75 to 76 and SPL from 64 to 68.
- Self-Active Learning adds 6.92 SR points on REVERIE val unseen over the MEO-only variant (63.70 to 68.11 SR for DUET).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ATENA to DUET | DUET · REVERIE Val Unseen | SR: 63.70 -> 68.11 | Helped |
| Enable Self-Active Learning (SAL) (ablation) | DUET + MEO · REVERIE Val Unseen | SR: 63.70 -> 68.11 | Helped |
| Enable Self-Active Learning (SAL) (ablation) | DUET + MEO · REVERIE Val Unseen | SPL: 42.49 -> 45.82 | Helped |
| Enable Self-Active Learning (SAL) (ablation) | DUET + MEO · REVERIE Val Unseen | RGSPL: 27.83 -> 31.26 | Helped |
| Apply ATENA to GOAT | GOAT · REVERIE Test Unseen | SR: +7.47% | Helped |
| Apply ATENA to BEVBert | BEVBert · R2R-CE Val Seen | SPL: +6.7% | Helped |
Surprising findings
- FSTTA gave minimal or negative gains over the base policy on several REVERIE splits, and TENT and FSTTA sometimes hurt performance.
- Adding active learning to FSTTA gave minimal benefit, which the authors attribute to conflicts between its gradient modification and human guidance.
Limitations
- Generalizability beyond VLN (e.g., visual or object-goal navigation) is unexplored.
- Self-prediction head accuracy is 82.19% on REVERIE val unseen, so self-labels contain errors.
- Relies on human episodic success/failure feedback for uncertain episodes.
Experimental setup
Datasets: REVERIE R2R R2R-CE
Benchmarks: REVERIE R2R R2R-CE
Metrics: SR OSR SPL RGSPL TL NE
Baselines: Tent FSTTA HAMT DUET BEVBert ETPNav GOAT
Builds on: Tent FSTTA entropy minimization uncertainty sampling active learning
Closest papers by meaning
- AdaNav: Adaptive Reasoning with Uncertainty for Vision-Language Navigation (ICML 2026 · similarity 0.89)
- Test-Time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning (ICML 2025 · similarity 0.89)
- Vision-Language Navigation with Energy-Based Policy (NeurIPS 2024 · similarity 0.88)
- CompassNav: Steering From Path Imitation to Decision Understanding In Navigation (ICLR 2026 · similarity 0.88)
- ACTIVE-o3 : Empowering MLLMs with Active Perception via Pure Reinforcement Learning (ICML 2026 · similarity 0.87)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.