Research map / Reinforcement learning / Visual and representation RL
Versatile Navigation Under Partial Observability via Value-guided Diffusion Policy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a value-guided diffusion policy with a differentiable QMDP-style planner, best-plan backtracking, and BEV projection of RGB-D point clouds enabling zero-shot 2D-to-3D navigation under partial observability.
Problem
Route planning and navigation under partial observability, where autoregressive planners are myopic and diffusion-based behavior cloning fails in unseen scenarios and lacks value guidance.
Approach
Combines a diffusion plan generator conditioned on partial maps with a learned QMDP-based value function (state estimation plus differentiable value iteration with valid-action masking), selects plans by multi-step value with a best-plan backtracking buffer, and transfers 2D policies to 3D by projecting segmented RGB-D point clouds onto BEV grid maps.
Key results
- On 15x15 GridMaze2D (view range 2) the method reaches 0.906±0.010 success vs. 0.855 for CALVIN and 0.060 for Diffusion Policy.
- On 30x30 mazes (view range 2) success is 0.624±0.032 vs. 0.326 for CALVIN and 0.000 for Diffusion Policy.
- On AVD, zero-shot transfer from 2D achieves 0.679 mean success vs. 0.635 for CALVIN-2D; retrained with RGB reaches 0.706 vs. 0.682 for CALVIN-3D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use value-guided diffusion planning with best-plan selection | GridMaze2D navigation · GridMaze2D 15×15 (vr=2) | success rate: 0.855±0.030 -> 0.906±0.010 | Helped |
| use value-guided diffusion planning with best-plan selection | GridMaze2D navigation · GridMaze2D 30×30 (vr=2) | success rate: 0.326±0.030 -> 0.624±0.032 | Helped |
| transfer the 2D policy zero-shot using point-cloud projection | AVD navigation · AVD mean | success rate: 0.682±0.047 -> 0.679±0.040 | No clear effect |
| retrain with additional RGB inputs | Ours · AVD mean | success rate: 0.679±0.040 -> 0.706±0.032 | Helped |
| use multiple samples and voting for plan selection instead of single sampling (ablation) | Diffusion Policy · GridMaze2D 15×15 (vr=2) | success rate: 0.060±0.022 -> 0.114±0.025 | Helped |
| use multiple samples and voting for plan selection instead of single sampling (ablation) | Diffusion Policy · AVD Home_001_1 | success rate: 0.024±0.012 -> 0.082±0.026 | Helped |
| add value guidance to multi-sampling and voting (ablation) | Diffusion Policy · GridMaze2D 15×15 (vr=2) | success rate: 0.114±0.025 -> 0.538±0.010 | Helped |
| add value guidance to multi-sampling and voting (ablation) | Diffusion Policy · AVD Home_001_1 | success rate: 0.082±0.026 -> 0.542±0.031 | Helped |
| add best-plan memory and backtracking (ablation) | Value-guided diffusion policy · GridMaze2D 15×15 (vr=2) | success rate: 0.538±0.010 -> 0.906±0.010 | Helped |
| add best-plan memory and backtracking (ablation) | Value-guided diffusion policy · AVD Home_001_1 | success rate: 0.542±0.031 -> 0.776±0.028 | Helped |
| remove the point-cloud-to-BEV projector (ablation) | Value-guided diffusion policy · AVD Home_001_1 | success rate: 0.776±0.028 -> 0.486±0.036 | Hurt |
Surprising findings
- Diffusion Policy (pure behavior cloning) fails almost completely under partial observability, reaching 0.000 success on 30x30 mazes.
- Replacing the value-based plan selection with majority voting over multi-sampled plans gives only a small gain (0.060 to 0.114), while value guidance raises it to 0.538.
Limitations
- Performance declines noticeably in larger mazes (30x30 success 0.624) due to longer planning horizons.
- The 3D evaluation relies on a semantic segmentation pipeline and is limited to AVD scenes, with a small number of scenes for retraining.
Experimental setup
Datasets: GridMaze2D (synthetic random mazes) Active Vision Dataset (AVD)
Benchmarks: GridMaze2D Active Vision Dataset embodied object search
Metrics: success rate
Baselines: CALVIN Diffusion Policy CALVIN-3D CALVIN-2D
Builds on: Value Iteration Networks CALVIN Diffusion Policy QMDP Diffuser
Closest papers by meaning
- Trajectory Diffusion for ObjectGoal Navigation (NeurIPS 2024 · similarity 0.87)
- Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving (CVPR 2024 · similarity 0.86)
- Diffusion-Based Planning for Autonomous Driving with Flexible Guidance (ICLR 2025 · similarity 0.86)
- Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction (ICLR 2024 · similarity 0.86)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.86)
- ChainFlow-VLA: Causal Flow Planning with Vision-Language Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.