Research map / Vision-language and multimodal / Robot action and navigation
Learning Navigational Visual Representations with Semantic Map Supervision
Yicong Hong, Yang Zhou, Ruiyi Zhang, Franck Dernoncourt, Trung Bui, Stephen Gould, Hao Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Ego2-Map contrastive learning to the frozen CLIP baseline (ablation) | CLIP-ViT-B/16 RGBD encoder · R2R-CE Val-Unseen (A_High) | SPL: 42.55 -> 45.89 | Improved |
| add Ego2-Map contrastive loss to angular and explorable objectives (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_High) | SPL: 37.25 -> 46.11 | Improved |
| train angular offset prediction as the only objective (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_High) | SPL: 42.55 -> 13.65 | Hurt |
| add angular offset prediction to explorable plus contrastive objectives (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_High) | SPL: 45.98 -> 46.11 | No clear effect |
| add explorable distance prediction to contrastive objective (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_Low) | Collision rate: 13.19 -> 10.09 | Improved |
| reduce number of pre-training environments to 50% (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_High) | SPL: 46.11 -> 42.24 | Hurt |
| pre-train on only the downstream MP3D environments instead of HM3D (ablation) | Ego2-Map pre-training · R2R-CE Val-Unseen (A_High) | SPL: 46.11 -> 35.76 | Hurt |
| initialize RGBD encoder randomly instead of from CLIP (ablation) | Ego2-Map RGBD encoder · R2R-CE Val-Unseen (A_High) | SPL: 46.11 -> 43.89 | Hurt |
| encode maps with IN-21K ViT instead of CLIP ViT (ablation) | Ego2-Map map encoder · R2R-CE Val-Unseen (A_High) | SPL: 45.73 -> 42.01 | Hurt |
| apply Ego2-Map objectives to CLIP-ViT+ResNet-50 depth encoder | CLIP-ViT+Depth · R2R-CE Val-Unseen (A_Low) | SPL: 21.69 -> 28.74 | Improved |
| keep object semantics in the map (ablation) | Ego2-Map contrastive learning · R2R-CE Val-Unseen (A_High) | SPL: 42.25 -> 46.11 | Improved |
| keep target view in the view pair (ablation) | Ego2-Map contrastive learning · R2R-CE Val-Unseen (A_High) | SPL: 41.21 -> 46.11 | Improved |
Closest papers by meaning
- Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation (NeurIPS 2025 · similarity 0.89)
- Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction (ICLR 2024 · similarity 0.88)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.88)
- Imagine Before Go: Self-Supervised Generative Map for Object Goal Navigation (CVPR 2024 · similarity 0.87)
- Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration (AAAI 2025 · similarity 0.87)
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
