Research map / Multi-agent systems and games / Human-AI and goal-conditioned
MO-DDN: A Coarse-to-Fine Attribute-based Exploration Agent for Multi-Object Demand-driven Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MO-DDN, a multi-object demand-driven navigation benchmark with basic and preferred demands, and C2FAgent, a coarse-to-fine attribute-based modular exploration agent using a VQ-VAE-style attribute model.
Problem
Demand-driven navigation assumes one object satisfies each demand and ignores user preferences; real demands often need multiple objects and personalized choices.
Approach
Extends DDN's attribute features to multi-object settings via a discrete codebook (initialized from CLIP features) with matching and reconstruction losses, and uses a two-phase agent: block-level attribute-similarity waypoint selection (coarse) followed by an imitation-learned end-to-end reporting policy (fine), with weighted basic/preferred scores.
Key results
- C2FAgent (LLM branch) reaches 23.82 SR_b and 14.28 SR_p in seen scene/seen task, versus 18.61 SR_b and 13.25 SR_p for MOPA+LLM (which uses ground-truth labels).
- On unseen scene/unseen task, C2FAgent (LLM branch) achieves 15.88 SR_b and 9.48 SR_p, outperforming DDN (7.10 SR_b, 4.70 SR_p).
- Ablation: replacing the attribute-based coarse exploration with FBE, LLM, or CLIP scoring lowers SR_b to 14.56, 13.69, and 12.51 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace attribute-based coarse exploration with FBE (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 14.56 (5.57) | Hurt |
| replace attribute-based coarse exploration with LLM waypoint selection (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 13.69 (4.91) | Hurt |
| replace attribute-based coarse exploration with CLIP feature similarity (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 12.51 (3.85) | Hurt |
| replace the fine exploration module with ZSON (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 8.74 (5.07) | Hurt |
| replace the fine exploration module with VTN (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 16.89 (4.57) | Hurt |
| replace the fine exploration module with random actions (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 5.82 (1.22) | Hurt |
| add VQ-VAE losses to attribute training (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 18.34 (1.33) -> 23.82 (3.89) | Helped |
| initialize the codebook (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 17.36 (2.32) -> 23.82 (3.89) | Helped |
| increase preferred-score weight to r_p=2 (r_b=1) (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_p: 14.28 (3.72) -> 15.05 (1.58) | Helped |
| increase preferred-score weight to r_p=2 (r_b=1) (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 19.89 (2.00) | Hurt |
| set preferred-score weight to r_p=0 (r_b=1) (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_b: 23.82 (3.89) -> 25.10 (2.06) | Helped |
| set preferred-score weight to r_p=0 (r_b=1) (ablation) | C2FAgent · MO-DDN, seen scene, seen task | SR_p: 14.28 (3.72) -> 12.43 (1.04) | Hurt |
Surprising findings
- Increasing the preferred-score weight trades off basic success for preferred success (r_b=1, r_p=0 gives highest SR_b of 25.10 but lowest SR_p of 12.43).
Limitations
- Assumes a fixed number of attributes per instruction and object (k1, k2).
- Evaluated only in HSSD scenes with a limited set of 358 object categories.
- Find action reports objects within a distance threshold rather than requiring bounding boxes, simplifying the task.
Experimental setup
Datasets: HSSD MO-DDN tasks generated with GPT-4 GPT-4-generated language-grounding tasks
Benchmarks: MO-DDN
Metrics: SR_b SR_p SPL_b SPL_p
Baselines: Random VTN ZSON DDN MOPA+LLM FBE+LLM
Builds on: Demand-driven Navigation (DDN) VQ-VAE CLIP Multi-object Navigation (MON)
Closest papers by meaning
- TP-MDDN: Task-Preferenced Multi-Demand-Driven Navigation with Autonomous Decision-Making (NeurIPS 2025 · similarity 0.89)
- Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences (CVPR 2024 · similarity 0.85)
- Reasoning in Visual Navigation of End-to-end Trained Agents: A Dynamical Systems Approach (CVPR 2025 · similarity 0.84)
- Seeing through Uncertainty: Robust Task-Oriented Optimization in Visual Navigation (NeurIPS 2025 · similarity 0.84)
- Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction (ICLR 2025 · similarity 0.84)
- Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.