Research map / Detection and segmentation / Open-set and few-shot detection
GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAMap, a zero-shot object goal navigation method that scores observations with multi-scale CLIP patch similarity to LLM-generated geometric-part and affordance attributes and builds a 2D value map for frontier-style exploration.
Problem
Zero-shot object goal navigation struggles when target objects are only partially observed and categorical semantic maps lack geometric and functional information about the environment.
Approach
Replace categorical semantic maps with a Geometric-part and Affordance Map: an LLM (GPT-4V) proposes object parts and affordances, CLIP scores multi-scale image patches against these attributes, scores are averaged across scales and max-merged into a 2D grid, and FMM navigates to the highest-scoring region.
Key results
- GAMap reaches 53.1% SR and 26.0% SPL on HM3D, versus 42.0% SR for VoroNav among methods without locomotion or semantic training.
- On Gibson, GAMap achieves 85.7% SR and 55.5% SPL, exceeding VLFM's 84.0% SR and 52.2% SPL in the main table.
- In a comparison with VLFM's mapping kept but FMM path planning, GAMap outperforms by 4.32% SR and 10.17% SPL on HM3D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use geometric-part and affordance guidance instead of categorical semantic guidance (ablation) | GAMap · Navigation evaluation | Detection Error: 16.95 -> 9.75 | Helped |
| Use geometric-part and affordance guidance instead of categorical semantic guidance (ablation) | GAMap · Navigation evaluation | Planning Error: 19.1 -> 11.34 | Helped |
| Use geometric-part and affordance guidance instead of categorical semantic guidance (ablation) | GAMap · Navigation evaluation | Exploration Error: 15.65 -> 12.78 | Helped |
| Increase geometric parts from 0 to 3 with no affordance attributes (ablation) | GAMap · HM3D mini-validation split | SR: — -> 47.5 | Helped |
| Increase affordance attributes from 0 to 3 with no geometric parts (ablation) | GAMap · HM3D mini-validation split | SR: — -> 44.0 | Helped |
| Use 3 affordance attributes and 3 geometric parts (ablation) | GAMap · HM3D mini-validation split | SR: 50.3 | Helped |
| Increase scale levels from 1 to 4 (ablation) | GAMap · HM3D mini-validation split | SR: approximately 10% | Helped |
| Increase scale levels from 1 to 4 (ablation) | GAMap · HM3D mini-validation split | SPL: about 20% | Helped |
| Replace patch-based scoring with gradient-based scoring (ablation) | GAMap · GA score calculation | SR: 50.25 -> 49.78 | Hurt |
| Replace patch-based scoring with gradient-based scoring (ablation) | GAMap · GA score calculation | time: 0.7 -> 0.9 | Hurt |
| Use BLIP-2 instead of CLIP as the patch scoring encoder (ablation) | GAMap · GA score calculation | time: 0.7 -> 1.6 | Hurt |
| Use max instead of average to aggregate scores across scales (ablation) | GAMap · GA score calculation | SR: 50.25 -> 49.75 | Hurt |
Surprising findings
- Gradient-based GA score computation attends to irrelevant regions such as the ceiling, and is slower than the patch-based method.
- The best reported HM3D SPL (26.0%) is below VLFM's 30.4%, which the authors attribute to VLFM's trained local policy.
Limitations
- Relies heavily on CLIP's visual processing and the accuracy of LLM-inferred geometric parts and affordances.
- Multi-scale scoring adds complexity and computational overhead (about 2 FPS).
- Real-world evaluation is only qualitative, shown as a video demo.
Experimental setup
Datasets: HM3D Gibson
Benchmarks: HM3D Gibson
Metrics: Success Rate SPL FPS
Baselines: SemExp ZSON PixNav VLFM PONI FbE L3MVN CoW ESC SemUtil VoroNav
Builds on: VLFM CLIP Fast Marching Method Frontier-based exploration
Closest papers by meaning
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation (NeurIPS 2025 · similarity 0.89)
- Affordances-Oriented Planning Using Foundation Models for Continuous Vision-Language Navigation (AAAI 2025 · similarity 0.87)
- SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation (NeurIPS 2024 · similarity 0.87)
- LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation (NeurIPS 2026 · similarity 0.87)
- VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model (ICML 2024 · similarity 0.86)
- An Interactive Navigation Method with Effect-oriented Affordance (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.