Research map / Vision-language and multimodal / Robot action and navigation
Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FSTTA, a fast-slow test-time adaptation method for online vision-and-language navigation that adapts LayerNorm parameters via gradient and parameter decomposition-accumulation (SVD-based) analysis.
Problem
Online VLN agents must execute instructions sequentially across changing environments; frequent test-time updates cause instability and catastrophic forgetting, while infrequent updates fail to adapt, creating an adaptability-stability dilemma.
Approach
A unified decomposition-accumulation framework applied to both gradients (fast phase, every M action steps, with a gradient-covariance-based dynamic learning rate) and parameter trajectories (slow phase, every N samples, reverting to historical states), using entropy minimization on the last four LayerNorm layers.
Key results
- On REVERIE Val Unseen, DUET-FSTTA improves SR from 46.98 to 54.15 and SPL from 33.73 to 36.41.
- Ablation on REVERIE Val Unseen: integrating fast and slow phases progressively raises SR by 2.8% and 4.3% over the Tent-based baseline.
- On SOON Test Unseen, DUET SPL improves from 21.42 to 23.23 with FSTTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply fast-slow test-time adaptation | DUET · REVERIE Val Unseen | SR: 46.98 -> 54.15 | Helped |
| Apply fast-slow test-time adaptation | DUET · REVERIE Val Unseen | SPL: 33.73 -> 36.41 | Helped |
| Apply fast-slow test-time adaptation | HM3D · REVERIE Val Unseen | SR: 55.89 -> 57.02 | Helped |
| Apply fast-slow test-time adaptation | DUET · R2R Val Unseen | SR: 72% rightarrow 75% | Helped |
| Apply fast-slow test-time adaptation | HM3D · R2R Val Unseen | SPL: 62% -> 63% | Helped |
| Apply fast-slow test-time adaptation | HM3D · SOON Val Unseen | SR: 41.00 -> 42.44 | Helped |
| Apply fast-slow test-time adaptation | DUET · SOON Test Unseen | SPL: 21.42% -> 23.23% | Helped |
| Apply fast-slow test-time adaptation | DUET · R2R-CE Val Unseen | SR: 46 -> 48 | Helped |
| Add fast update to DUET (ablation) | DUET · REVERIE Val Unseen | SR: 46.98 -> 49.74 | Helped |
| Add dynamic learning-rate scaling to fast update (ablation) | DUET fast update · REVERIE Val Unseen | SR: 49.74 -> 49.82 | Helped |
| Add slow update to fast update with DLR (ablation) | DUET fast update with DLR · REVERIE Val Unseen | SR: 49.82 -> 54.15 | Helped |
| Use TENT interval 3 instead of base DUET (ablation) | DUET · REVERIE Val Unseen | SR: 46.98 -> 48.60 | Helped |
Surprising findings
- Directly applying existing TTA methods (EATA, CoTTA, NOTE, SAR, ViDA) to online VLN yields no significant gains, and overly frequent or overly infrequent Tent updates both underperform.
Limitations
- Only adapts LayerNorm affine parameters, so other layer types are not updated.
- Online setting is simulated by shuffling and sequentially feeding test samples rather than a real online dataset.
- TTA adds computational overhead and update frequencies are fixed rather than adaptive.
Experimental setup
Datasets: REVERIE R2R SOON R2R-CE
Benchmarks: REVERIE R2R SOON R2R-CE
Metrics: SR SPL OSR RGSPL TL NE RGS execution time
Baselines: DUET Tent EATA CoTTA NOTE SAR ViDA BEVBert HM3D WS-MGMap GridMM
Builds on: TENT DUET PCA
Closest papers by meaning
- Test-Time Adaptation for Online Vision-Language Navigation with Feedback-based Reinforcement Learning (ICML 2025 · similarity 0.91)
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.87)
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.86)
- DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.86)
- Active Test-time Vision-Language Navigation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.