Research map / Multi-agent systems and games / Human-AI and goal-conditioned
GOALNET: Interleaving Neural Goal Predicate Inference with Classical Planning for Generalization in Robot Instruction Following
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GOALNET, a neuro-symbolic method that iteratively predicts a single positive and negative subgoal predicate from the world state and instruction, and executes classical planner (Rintanen) plans to follow natural language robot instructions.
Problem
Enabling robots to sequence actions for high-level natural-language instructions with multi-stage tasks, generalizing to novel instructions and unseen objects, where direct conjunctive goal prediction is exponential and sensitive to side-effect predicates.
Approach
Interleave a neural subgoal-predicate predictor (one positive and one negative predicate per step, using object-centric graph encoding, instruction-conditioned attention, and PDDL grammar masks) with a classical planner and simulator execution in a loop; GOALNET* uses factored per-object scoring to handle unseen objects.
Key results
- GOALNET reaches GRR 0.499 vs 0.370 for next best (SHE&CHAI) and 0.182 for TANGO on the main benchmark; training with Rintanen raises GRR to 0.658.
- On unseen-object test set, GOALNET* reaches GRR 0.196 vs 0.130 for GOALNET, and GOALNET* outperforms baselines in generalization to unseen objects (up to 1.36x claimed).
- Ablating negative subgoal prediction drops GRR by 5 points; ablating positive subgoal prediction drops GRR by 37 points, and removing temporal context drops GRR by 31 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use interleaved subgoal prediction, planning, and execution | GOALNET · Benchmark test | GRR: 0.370 -> 0.499 | Helped |
| Replace the fixed-object predictor with factored object-score prediction | GOALNET · Unseen objects | GRR: 0.130 -> 0.196 | Helped |
| Add relational information to the world-state input (ablation) | GOALNET · Benchmark test | GRR: 0.459 -> 0.499 | Helped |
| Add instance grounding of objects (ablation) | GOALNET · Benchmark test | GRR: 0.445 -> 0.499 | Helped |
| Add negative-subgoal prediction (ablation) | GOALNET · Benchmark test | GRR: 0.450 -> 0.499 | Helped |
| Add positive-subgoal prediction (ablation) | GOALNET · Benchmark test | GRR: 0.134 -> 0.499 | Helped |
| Add temporal context encoding (ablation) | GOALNET · Benchmark test | GRR: 0.188 -> 0.499 | Helped |
| Add the PDDL-based grammar mask (ablation) | GOALNET · Benchmark test | GRR: 0.459 -> 0.499 | Helped |
| Train with RINTANEN instead of SYMSIM (ablation) | GOALNET · Benchmark test | GRR: 0.499 -> 0.658 | Helped |
| Use factored object-score prediction (ablation) | GOALNET · Benchmark test | GRR: 0.499 -> 0.377 | Hurt |
| Use GOALNET for verb replacement generalization | GOALNET · Verb replacement | GRR: 0.138 -> 0.236 | Helped |
| Use GOALNET for paraphrased instructions | GOALNET · Paraphrasing | GRR: 0.128 -> 0.303 | Helped |
Surprising findings
- GPT-3.5 Turbo with ProgPrompt-style few-shot prompting performs worst among compared methods (GRR 0.126).
Limitations
- Struggles when multiple plausible subgoals exist, causing imperfect scores.
- Assumes cause-effect modeling is given as PDDL input.
- Rare loops where the subgoal predicts the same goal repeatedly (7 of 181 plans).
Experimental setup
Datasets: Misra et al. (2015) kitchen and living-room instruction-trajectory benchmark ConceptNet embeddings SentenceBERT
Benchmarks: Misra et al. (2015) instruction-following benchmark verb replacement generalization set paraphrasing generalization set unseen objects generalization set
Metrics: SJI IED F1 GRR
Baselines: SHE&CHAI TANGO PIPELINE GN-SYMSIM GAT GPT-3.5 TURBO
Builds on: TANGO She and Chai goal inference Rintanen planner ProgPrompt
Closest papers by meaning
- $\textit{Hyper-GoalNet}$: Goal-Conditioned Manipulation Policy Learning with HyperNetworks (NeurIPS 2025 · similarity 0.89)
- GoalLadder: Incremental Goal Discovery with Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning (ICLR 2025 · similarity 0.88)
- GNN-Transformer Task Planning Enhanced with Semantic-Driven Data Augmentation (AAAI 2025 · similarity 0.88)
- Embodied Task Planning via Graph-Informed Action Generation with Large Language Models (ICML 2026 · similarity 0.86)
- Generalizing Verifiable Instruction Following (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.