Research map / Detection and segmentation / Open-set and few-shot detection
Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IEVE, a modular Exploration-Verification-Exploitation navigation framework that switches among three goal-mapping modes to verify instance identity while approaching, reaching 0.684/0.702 Success on HM3D-SEM Instance ImageGoal Navigation.
Problem
Instance ImageGoal Navigation (IIN): navigate to a specific object instance shown in a goal image while rejecting similar distractors of the same class from different viewpoints in unexplored environments.
Approach
Treats target confirmation as a sequential decision: a switch policy chooses among exploration, verification (approach to check a potential target via keypoint matching and distance), and exploitation (confirmed target), with a switch function using Euclidean distance and matched LightGlue keypoints as adaptive thresholds.
Key results
- IEVE with Mask R-CNN reaches 0.684 Success vs. 0.561 for Mod-IIN on HM3D-SEM validation IIN.
- IEVE with InternImage segmentation reaches 0.702 Success and 0.252 SPL.
- Replacing the verification-switch policy with a fixed exploration-exploitation threshold drops Success from 0.666 to 0.590 with Mask R-CNN and RL exploration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IEVE Mask RCNN instead of Mod-IIN | Instance ImageGoal Navigation · HM3D-SEM validation | Success: 0.561 -> 0.684 | Helped |
| Use IEVE InternImage instead of Mod-IIN | Instance ImageGoal Navigation · HM3D-SEM validation | Success: 0.561 -> 0.702 | Helped |
| Fine-tune OVRL-v2 on IIN | OVRL-v2 · HM3D-SEM validation | Success: 0.006 -> 0.248 | Helped |
| Add a verification stage by using EVE instead of EE (ablation) | IEVE with Mask RCNN · HM3D-SEM validation | Success: 0.590 -> 0.666 | Helped |
| Add a verification stage by using EVE instead of EE (ablation) | IEVE with SwinTrans classification and Mask RCNN perception · HM3D-SEM validation | Success: 0.620 -> 0.684 | Helped |
| Replace RL exploration with frontier exploration (ablation) | IEVE with Mask RCNN · HM3D-SEM validation | SPL: 0.235 -> 0.182 | Hurt |
| Replace RL exploration with random exploration (ablation) | IEVE with Mask RCNN · HM3D-SEM validation | Success: 0.666 -> 0.654 | Hurt |
| Replace segmentation-based classification with SwinTrans classification (ablation) | IEVE with Mask RCNN perception · HM3D-SEM validation | Success: 0.666 -> 0.684 | Helped |
| Replace segmentation-based classification with SwinTrans classification (ablation) | IEVE with InternImage perception · HM3D-SEM validation | Success: 0.702 -> 0.676 | Hurt |
| Use InternImage rather than Mask RCNN for perception (ablation) | IEVE · HM3D-SEM validation | Success: 0.666 -> 0.702 | Helped |
| Replace Mask RCNN perception with oracle segmentation and use ground-truth classification (ablation) | IEVE · HM3D-SEM validation | Success: 0.682 -> 0.850 | Helped |
| Use RL exploration rather than frontier exploration (ablation) | IEVE with Mask RCNN · HM3D-SEM validation | Success: 0.676 -> 0.666 | No clear effect |
Surprising findings
- Frontier exploration achieves higher Success (0.676) than RL exploration (0.666) with Mask R-CNN but much lower SPL (0.182 vs. 0.235).
- Adding SwinTransformer instance classification helps Mask R-CNN but suppresses InternImage performance (0.676 vs. 0.702).
Limitations
- End-to-end RL baseline generalizes poorly and needs 3.5 billion training steps.
- Target identification at long distances is unreliable; true-positive rate drops sharply for hard difficulty levels.
Experimental setup
Datasets: HM3D-SEM Instance ImageGoal Navigation episode dataset (Krantz et al.) COCO
Benchmarks: HM3D-SEM IIN validation split
Metrics: Success SPL
Baselines: RL end-to-end agent OVRL-v2 Mod-IIN Frontier exploration Random exploration
Builds on: Mod-IIN OVRL-v2 LightGlue DISK Mask R-CNN
Closest papers by meaning
- Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments (NeurIPS 2024 · similarity 0.87)
- End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon (ICLR 2024 · similarity 0.87)
- Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator (NeurIPS 2026 · similarity 0.86)
- REGNav: Room Expert Guided Image-Goal Navigation (AAAI 2025 · similarity 0.86)
- FlySearch: Exploring how vision-language models explore (NeurIPS 2025 · similarity 0.85)
- Imagine Before Go: Self-Supervised Generative Map for Object Goal Navigation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.