Research map / Vision-language and multimodal / Robot action and navigation
BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BeliefMapNav, a zero-shot object navigation system that builds a 3D voxel belief map from hierarchical CLIP semantics and GPT-4o landmarks, then plans frontier visits by minimizing expected search cost.
Problem
Zero-shot object navigation in unknown 3D environments, where LLM/VLM greedy goal selection lacks global spatial understanding and produces inefficient back-and-forth search.
Approach
A 3D voxel belief map fuses multi-scale hierarchical CLIP semantic features with LLM-generated room/region/object landmarks to give a prior over target presence; a visibility map updates it into a posterior, and frontiers are ordered by simulated-annealing optimization of expected path cost rather than greedily.
Key results
- On HM3D, BeliefMapNav reaches 61.4 SR and 30.6 SPL, the best SR among compared methods (InstructNav 58.0 SR).
- On MP3D, reaches 37.3 SR and 17.6 SPL; on HSSD, 65.2 SR and 32.1 SPL.
- Ablation on 400 HM3D episodes: removing the visibility map drops SR from 62.5 to 57.2 and SPL from 31.6 to 28.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove the belief-based planner (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SR: 62.5 -> 56.0 | Hurt |
| remove the belief-based planner (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SPL: 31.6 -> 29.3 | Hurt |
| remove the visibility map (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SR: 62.5 -> 57.2 | Hurt |
| remove the visibility map (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SPL: 31.6 -> 28.0 | Hurt |
| replace CLIP with BLIP (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SR: 62.5 -> 59.3 | Hurt |
| replace CLIP with BLIP (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SPL: 31.6 -> 31.0 | Hurt |
| replace CLIP with BLIP-2 (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SR: 62.5 vs. 62.0 | No clear effect |
| replace CLIP with BLIP-2 (ablation) | BeliefMapNav · HM3D validation, 400 episodes | SPL: 31.6 vs. 31.1 | No clear effect |
| add room, region, and object-level landmarks (ablation) | BeliefMapNav without landmarks · HM3D validation, 400 episodes | SR: 60.0 -> 62.5 | Helped |
| add room, region, and object-level landmarks (ablation) | BeliefMapNav without landmarks · HM3D validation, 400 episodes | SPL: 30.9 -> 31.6 | Helped |
| add scene-, region-, and object-level semantics (ablation) | BeliefMapNav with scene- and region-level semantics · HM3D validation, 400 episodes | SR: 61.5 -> 62.5 | Helped |
| add object-level semantics (ablation) | BeliefMapNav with scene- and region-level semantics · HM3D validation, 400 episodes | SPL: 32.0 -> 31.6 | Hurt |
Surprising findings
- Omitting object-level semantics yields higher SPL (32.0) than the full hierarchy (31.6), though with lower SR.
- Most HM3D failures (36.78%) come from targets on a different floor than the start, rather than exploration failure.
Limitations
- Validated only on object navigation, not mobile manipulation or real-world deployment.
- Local planner limits performance in narrow areas; floor changes and stair traversal fail.
- MP3D mesh holes let the agent mistake see-through regions for targets, limiting gains on MP3D.
Experimental setup
Datasets: HM3D MP3D HSSD
Benchmarks: HM3D MP3D HSSD
Metrics: SR SPL
Baselines: ZSON PSL PixNav VLFM ESC CoW L3MVN ImagineNav VoroNav GAMap OpenFMNav InstructNav
Builds on: VLFM CoW InstructNav SAM
Closest papers by meaning
- GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidance (NeurIPS 2024 · similarity 0.89)
- Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation (NeurIPS 2025 · similarity 0.89)
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models (AAAI 2024 · similarity 0.88)
- VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model (ICML 2024 · similarity 0.88)
- RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation (NeurIPS 2026 · similarity 0.88)
- UniGoal: Towards Universal Zero-shot Goal-oriented Navigation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.