Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Navigating to Objects Specified by Images
Jacob Krantz, Theophile Gervet, Karmesh Yadav, Austin Wang, Chris Paxton, Roozbeh Mottaghi, Dhruv Batra, Jitendra Malik, Stefan Lee, Devendra Singh Chaplot
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use modular zero-fine-tuning system instead of end-to-end RL baseline | Mod-IIN · HM3D InstanceImageNav (Val) | SR: 0.083 -> 0.561 | Improved |
| use modular system instead of fine-tuned OVRL-v2 | Mod-IIN · HM3D InstanceImageNav (Val) | SR: 0.248 -> 0.561 | Improved |
| fine-tune pretrained OVRL-v2 on InstanceImageNav | OVRL-v2 · HM3D InstanceImageNav (Val) | SR: 0.006 -> 0.248 | Improved |
| threshold sum of match confidence rather than count of matched keypoints (ablation) | Mod-IIN re-identification · HM3D InstanceImageNav (Val) | SR: 0.539 -> 0.561 | Improved |
| use Detic instance mask instead of static central crop for goal localization (ablation) | Mod-IIN goal localization · HM3D InstanceImageNav (Val) | SR: 0.523 -> 0.561 | Improved |
| use keypoint-projection localization instead of class-based Detic-ObjectNav localization (ablation) | Mod-IIN goal localization · HM3D InstanceImageNav (Val) | SR: 0.488 -> 0.561 | Improved |
| use SuperPoint+SuperGlue keypoint Re-ID instead of ResNet features (ablation) | Mod-IIN re-identification · HM3D InstanceImageNav (Val) | SR: 0.138 -> 0.488 (both with Detic-ObjectNav) | Improved |
| use SuperPoint+SuperGlue keypoint Re-ID instead of CLIP features (ablation) | Mod-IIN re-identification · HM3D InstanceImageNav (Val) | SR: 0.097 -> 0.488 (both with Detic-ObjectNav) | Improved |
| replace predicted Re-ID with oracle Re-ID while keeping predicted localization (ablation) | Mod-IIN · HM3D InstanceImageNav (Val) | SR: 0.561 -> 0.498 | No clear effect |
| replace predicted goal localization with oracle localization (ablation) | Mod-IIN · HM3D InstanceImageNav (Val) | SR: 0.561 -> 0.647 | Improved |
| provide oracle Re-ID and oracle localization (ablation) | Mod-IIN · HM3D InstanceImageNav (Val) | SR: 0.561 -> 0.845 | Improved |
Closest papers by meaning
- Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigation (CVPR 2024 · similarity 0.89)
- Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments (NeurIPS 2024 · similarity 0.88)
- Imagine Before Go: Self-Supervised Generative Map for Object Goal Navigation (CVPR 2024 · similarity 0.88)
- End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon (ICLR 2024 · similarity 0.87)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.87)
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
