AI Paper Atlas

Research map / Multi-agent systems and games / Human-AI and goal-conditioned

Navigating to Objects Specified by Images

Jacob Krantz, Theophile Gervet, Karmesh Yadav, Austin Wang, Chris Paxton, Roozbeh Mottaghi, Dhruv Batra, Jitendra Malik, Stefan Lee, Devendra Singh Chaplot

ICCV 2023 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Vision-language navigation agents · 47 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use modular zero-fine-tuning system instead of end-to-end RL baselineMod-IIN · HM3D InstanceImageNav (Val)SR: 0.083 -> 0.561Improved
use modular system instead of fine-tuned OVRL-v2Mod-IIN · HM3D InstanceImageNav (Val)SR: 0.248 -> 0.561Improved
fine-tune pretrained OVRL-v2 on InstanceImageNavOVRL-v2 · HM3D InstanceImageNav (Val)SR: 0.006 -> 0.248Improved
threshold sum of match confidence rather than count of matched keypoints (ablation)Mod-IIN re-identification · HM3D InstanceImageNav (Val)SR: 0.539 -> 0.561Improved
use Detic instance mask instead of static central crop for goal localization (ablation)Mod-IIN goal localization · HM3D InstanceImageNav (Val)SR: 0.523 -> 0.561Improved
use keypoint-projection localization instead of class-based Detic-ObjectNav localization (ablation)Mod-IIN goal localization · HM3D InstanceImageNav (Val)SR: 0.488 -> 0.561Improved
use SuperPoint+SuperGlue keypoint Re-ID instead of ResNet features (ablation)Mod-IIN re-identification · HM3D InstanceImageNav (Val)SR: 0.138 -> 0.488 (both with Detic-ObjectNav)Improved
use SuperPoint+SuperGlue keypoint Re-ID instead of CLIP features (ablation)Mod-IIN re-identification · HM3D InstanceImageNav (Val)SR: 0.097 -> 0.488 (both with Detic-ObjectNav)Improved
replace predicted Re-ID with oracle Re-ID while keeping predicted localization (ablation)Mod-IIN · HM3D InstanceImageNav (Val)SR: 0.561 -> 0.498No clear effect
replace predicted goal localization with oracle localization (ablation)Mod-IIN · HM3D InstanceImageNav (Val)SR: 0.561 -> 0.647Improved
provide oracle Re-ID and oracle localization (ablation)Mod-IIN · HM3D InstanceImageNav (Val)SR: 0.561 -> 0.845Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy