Research map / Multi-agent systems and games / Human-AI and goal-conditioned
From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning
Honglin He, Yukai Ma, Brad Squicciarini, Wayne Wu, Bolei Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RL fine-tuning in simulation after offline pretraining (ablation) | S2E-BC · NavBench-GS Obstacle | SR: 0.42 -> 0.57 | Improved |
| add RL fine-tuning in simulation after offline pretraining (ablation) | S2E-BC · NavBench-GS Obstacle + Pedestrian | SR: 0.40 -> 0.51 | Improved |
| scale offline data from 250k to 750k samples (ablation) | S2E-BC · success rate (Figure 3a) | SR: +2% success rate | No clear effect |
| use RL instead of SFT for post-training with more training cost (ablation) | S2E · URBAN-SIM and NavBench-GS | SR | Improved |
| use anchor-guided multimodal GMM instead of single-mode regression (ablation) | S2E-BC · NavBench-GS Obstacle | SR: 0.33 -> 0.42 | Improved |
| replace anchor-guided GMM with discrete anchor classification (ablation) | S2E-BC · NavBench-GS Obstacle | SR: 0.42 -> 0.37 | Hurt |
| replace GMM decoder with diffusion (DiT) decoder (ablation) | S2E-BC · NavBench-GS Obstacle | SR: 0.42 -> 0.44 | No clear effect |
| add offline pretraining before RL (vs PPO from scratch) (ablation) | S2E · NavBench-GS Obstacle | SR: 0.02 -> 0.57 | Improved |
| increase number of anchors from 1 to 64 (ablation) | S2E · RECON test set | minADE: 0.21 -> 0.09 | Improved |
| use 16 anchors instead of 8 (ablation) | S2E · RECON test set | mAP: 0.62 -> 0.59 | Hurt |
| add RL fine-tuning of policy head | ViNT* · NavBench-GS | SR: 0.27 -> 0.39 | Improved |
| fine-tune residual attention module instead of action decoder layers with RL (ablation) | S2E RL fine-tuning · training stability | GPU memory: nearly 40GB -> 37GB | Improved |
Closest papers by meaning
- Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator (NeurIPS 2026 · similarity 0.88)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.87)
- Navigation World Models (CVPR 2025 · similarity 0.87)
- Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigation (CVPR 2025 · similarity 0.87)
- CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos (CVPR 2025 · similarity 0.87)
- EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
