Research map / Vision-language and multimodal / Robot action and navigation
Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically compares No-Think, Pre-Think and Post-Think reasoning for VLN-CE, finds test-time reasoning collapses, and proposes Aux-Think, which trains with CoT and instruction reconstruction as auxiliary tasks while acting without reasoning at test time, plus the R2R-CoT-320k dataset.
Problem
Explicit chain-of-thought reasoning at test time harms vision-language navigation performance in continuous environments, and reasoning strategies for VLN are unexplored.
Approach
Use CoT traces and instruction reconstruction only as auxiliary training tasks, co-trained with receding-horizon action prediction via prompt switching, while predicting actions directly (no reasoning) at test time.
Key results
- Aux-Think reaches 46.0 SR on R2R val-unseen with 0K extra data, versus NaVid 24.7 SR under the same R2R-CE-only setting.
- With extra data, Aux-Think achieves 54.8 SR on R2R val-unseen using 1600K samples, comparable to NaVILA (54.0 SR with 2770K).
- Under the R2R-CoT-320k-only comparison, Pre-Think reaches 11.4 SR and Post-Think 29.0 SR versus No-Think 35.1 SR, while Aux-Think reaches 41.3 SR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Aux-Think instead of No-Think direct action prediction | NVILA-lite 8B · R2R-CE Val-Unseen with R2R-CoT-320k training | SR: 35.1 -> 41.3 | Helped |
| Use Pre-Think instead of direct action prediction | NVILA-lite 8B · R2R-CE Val-Unseen with R2R-CoT-320k training | SR: 35.1 -> 11.4 | Hurt |
| Use Post-Think instead of direct action prediction | NVILA-lite 8B · R2R-CE Val-Unseen with R2R-CoT-320k training | SR: 35.1 -> 29.0 | Hurt |
| Add CoT reasoning as auxiliary supervision (ablation) | Aux-Think · R2R-CE Val-Unseen | SR: 35.1 -> 41.3 | Helped |
| Add instruction-based reasoning alongside CoT reasoning (ablation) | Aux-Think · R2R-CE Val-Unseen | SR: 41.3 -> 44.2 | Helped |
| Add receding-horizon action planning to CoT and instruction reasoning (ablation) | Aux-Think · R2R-CE Val-Unseen | SR: 44.2 -> 46.0 | Helped |
| Use instruction-based reasoning only instead of no auxiliary task (ablation) | Aux-Think · R2R-CE Val-Unseen | SR: 35.1 -> 40.6 | Helped |
| Use receding-horizon action planning only instead of no auxiliary task (ablation) | Aux-Think · R2R-CE Val-Unseen | SR: 35.1 -> 37.1 | Helped |
| Predict 3 actions rather than 1 in receding-horizon planning (ablation) | Aux-Think without CoT auxiliary task · R2R-CE Val-Unseen | SR: 35.1 -> 41.4 | Helped |
| Increase planning horizon from 3 to 4 predicted actions (ablation) | Aux-Think without CoT auxiliary task · R2R-CE Val-Unseen | SR: 41.4 -> 36.4 | Hurt |
| Increase planning horizon from 3 to 5 predicted actions (ablation) | Aux-Think without CoT auxiliary task · R2R-CE Val-Unseen | SR: 41.4 -> 37.1 | Hurt |
| Reduce Pre-Think CoT loss weight from 1 to 0.1 (ablation) | Pre-Think · R2R-CE Val-Unseen | SR: 11.4 -> 15.9 | Helped |
Surprising findings
- Test-time reasoning (Pre-Think and Post-Think) consistently underperforms direct action prediction, a phenomenon the authors term Test-time Reasoning Collapse.
- Pre-Think and Post-Think inference takes about 30s per step versus 1.25s for No-Think and Aux-Think.
Limitations
- Evaluated only under monocular RGB SFT on R2R; scaling to richer observations and larger datasets is untested.
- Aux-Think shows relatively weak NE and SPL, attributed to broader exploration.
- CoT annotations were generated by a VLM (Qwen2.5-VL-72B) with manual review, and the dataset release status is unclear.
Experimental setup
Datasets: R2R-CE RxR-CE R2R-CoT-320k
Benchmarks: R2R-CE val-unseen RxR-CE val-unseen
Metrics: NE OSR SR SPL average time
Baselines: No-Think Pre-Think Post-Think NaVid Uni-NaVid NaVILA BEVBert ETPNav ENP-ETPNav sim2real
Builds on: NaVILA NaVid Uni-NaVid Chain-of-Thought
Closest papers by meaning
- FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-and-Language Navigation (CVPR 2026 · similarity 0.93)
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · similarity 0.92)
- Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation (AAAI 2026 · similarity 0.90)
- Visual Planning: Let's Think Only with Images (ICLR 2026 · similarity 0.89)
- NavBench: Probing Multimodal Large Language Models for Embodied Navigation (NeurIPS 2025 · similarity 0.89)
- More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.