Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Large-scale real-robot and simulation study analyzing what a PPO-trained end-to-end point-goal navigation agent learns about dynamics, memory, planning, and value estimates.
Problem
Understanding what internal reasoning (dynamics models, memory, planning, value) end-to-end RL navigation agents learn when trained with realistic robot motion and evaluated on fast-moving real robots.
Approach
Reproduces and improves the dynamics-aware PointNav agent of Bono et al. and probes its recurrent memory and critic with a series of sensitivity, probing, memory-ablation, Shapley and expert-comparison experiments, showing Kalman-like prediction/correction, short-to-medium horizon pose prediction, and value estimates linked to replanning.
Key results
- Trained agent reaches 92.5% success rate on real Test-bldg/20 episodes with dynamics-aware training, versus 40% for the prior dynamics-free-style baseline D28-instant-like setup in the reproduction table.
- Latent memory linearly predicts future pose with 0.441m position error at 6s-horizon-equivalent (20 steps) for the linear + prev-action probe on 39k unseen-scene trajectories.
- Zeroing memory every 3s drops real success rate from 100% to 75%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a realistic dynamical model during training | D28 navigation agent · Test-bldg/20 (real) | SR: 10.0 -> 92.5 ±2.9 | Helped |
| add a realistic dynamical model during training | D28 navigation agent · HM3D/2.5k | SR: 27.6 -> 97.6 | Helped |
| zero latent memory periodically (ablation) | D28-dynamics · Test-bldg/14 | SR: 100.0 -> 85.0 (30 sec); 100.0 -> 80.0 (10 sec); 100.0 -> 75.0 (3 sec) | Hurt |
| retain latent memory instead of resetting it every 3 sec (ablation) | D28-dynamics · Test-bldg/14 | SR: 75.0 -> 100.0 | Helped |
| limit maximum velocity to 70% (ablation) | D28-dynamics · Test-bldg/14 | SR: 85.0 -> 100.0 | Helped |
| reduce maximum velocity to 50% (ablation) | D28-dynamics · Test-bldg/14 | SR: 100.0 -> 85.0 | Hurt |
| change observation-to-decision delay from 325 ms to 150 ms (ablation) | D28-dynamics · Test-bldg/14 | SR: 100.0 -> 100.0 | No clear effect |
| replace AMCL from 1D Lidar with R2D2 visual localization (ablation) | D28-dynamics · Test-bldg/14 | SR: 100.0 -> 42.9 | Hurt |
| add RGB test-time augmentation | D28-dynamics · real-world setup | SR: ↑15% | Helped |
| provide environment-parameter embeddings to the policy | D28-dynamics · HM3D/250 under dynamics changes | SR: +11%; +29%; +31% (response time); +11%; +23%; +28% (damping); +1%; +3%; +11% (max velocity) | Helped |
Surprising findings
- Visual localization with R2D2 replacing AMCL drops real success rate from 100% to 42.9%, showing localization does not emerge from RL loss alone.
- The agent fails to show evidence of correct long-term planning, exhibiting 'tunnel vision' in some cases.
Limitations
- Real-robot evaluation limited to 14-20 episodes per setting in one test building
- No evidence of correct long-term planning found
- Engineering improvements not ablated
Experimental setup
Datasets: HM3D HM3D val-mini (250 episodes) HM3D validation (2,500 episodes) Test-bldg real episodes
Benchmarks: HM3D/2.5k Test-bldg/20 (sim and real) HM3D/250
Metrics: success rate SPL SCT position error angular error occupancy accuracy
Baselines: Bono et al. [9] D28-dynamics baseline D28-instant D4 motion-command agent R2D2 visual localization Fast Marching Square expert planner
Builds on: PPO Bono et al. dynamics-aware navigation RMA Habitat
Closest papers by meaning
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · similarity 0.90)
- Visual Planning: Let's Think Only with Images (ICLR 2026 · similarity 0.89)
- Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction (ICLR 2024 · similarity 0.89)
- FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-and-Language Navigation (CVPR 2026 · similarity 0.89)
- Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation (NeurIPS 2025 · similarity 0.89)
- Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.