AI Paper Atlas

Research map / Vision-language and multimodal / Robot action and navigation

Learning Navigational Visual Representations with Semantic Map Supervision

Yicong Hong, Yang Zhou, Ruiyi Zhang, Franck Dernoncourt, Trung Bui, Stephen Gould, Hao Tan

ICCV 2023 · Vision-language and multimodal › Robot action and navigation · approach: Pretraining and data augmentation for VLN · 51 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Ego2-Map contrastive learning to the frozen CLIP baseline (ablation)CLIP-ViT-B/16 RGBD encoder · R2R-CE Val-Unseen (A_High)SPL: 42.55 -> 45.89Improved
add Ego2-Map contrastive loss to angular and explorable objectives (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_High)SPL: 37.25 -> 46.11Improved
train angular offset prediction as the only objective (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_High)SPL: 42.55 -> 13.65Hurt
add angular offset prediction to explorable plus contrastive objectives (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_High)SPL: 45.98 -> 46.11No clear effect
add explorable distance prediction to contrastive objective (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_Low)Collision rate: 13.19 -> 10.09Improved
reduce number of pre-training environments to 50% (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_High)SPL: 46.11 -> 42.24Hurt
pre-train on only the downstream MP3D environments instead of HM3D (ablation)Ego2-Map pre-training · R2R-CE Val-Unseen (A_High)SPL: 46.11 -> 35.76Hurt
initialize RGBD encoder randomly instead of from CLIP (ablation)Ego2-Map RGBD encoder · R2R-CE Val-Unseen (A_High)SPL: 46.11 -> 43.89Hurt
encode maps with IN-21K ViT instead of CLIP ViT (ablation)Ego2-Map map encoder · R2R-CE Val-Unseen (A_High)SPL: 45.73 -> 42.01Hurt
apply Ego2-Map objectives to CLIP-ViT+ResNet-50 depth encoderCLIP-ViT+Depth · R2R-CE Val-Unseen (A_Low)SPL: 21.69 -> 28.74Improved
keep object semantics in the map (ablation)Ego2-Map contrastive learning · R2R-CE Val-Unseen (A_High)SPL: 42.25 -> 46.11Improved
keep target view in the view pair (ablation)Ego2-Map contrastive learning · R2R-CE Val-Unseen (A_High)SPL: 41.21 -> 46.11Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy