Research map / Large language models / Agents, code and math
UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pre-trains a unified PDDL domain from 12,393 DROID manipulation demonstrations via energy-based keyframe extraction and closed-loop LLM/VLM domain generation, then fuses retrieved atomic domains into a task-specific meta-domain for zero-shot PDDL planning.
Problem
Robot task planning under implicit language and visual constraints needs realistic symbolic PDDL domains, but LLMs struggle to write domains for real-world manipulation and existing LLM-PDDL methods rely on handcrafted or narrow domains.
Approach
Learn many small atomic PDDL domains from real robot demonstration videos with closed-loop verification (solvability tests and commonsense plan checks), merge them into a unified knowledge graph, and fuse task-relevant subsets hierarchically into a compact meta-domain used for online planning.
Key results
- UniDomain reaches 85% task success and 83% optimality at K=0 on 100 unseen real-world tasks.
- Using atomic domains without fusion yields only 19% success, versus 85% for the fused meta-domain.
- Energy-based keyframe extraction takes 0.6 s per demonstration versus 47.8 s for similarity-based SigLIP-2 extraction, with 28% vs 15% single-pass domain success.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use energy-based rather than similarity-based keyframe extraction for one-shot domain generation (ablation) | UniDomain atomic domain learning · Atomic domain learning (40 DROID demonstrations) | success rate: 15% -> 28% | Helped |
| use energy-based rather than similarity-based keyframe extraction (ablation) | UniDomain · DROID demonstration processing | processing time per demonstration: 47.8 seconds to just 0.6 seconds | Helped |
| add domain fusion instead of selecting the closest atomic domain (ablation) | UniDomain · Evaluation tasks | success rate: 19% -> 85% | Helped |
| add LLM-based revision (ablation) | UniDomain atomic domain learning · Atomic domain learning (40 DROID demonstrations) | average refinement iterations: 1.36 -> 0.49 | Helped |
| remove closed-loop feedback mechanisms (ablation) | UniDomain atomic domain learning · Atomic domain learning | domain quality | Hurt |
| remove the solvability check (ablation) | UniDomain atomic domain learning · Atomic domain learning | domain quality | Hurt |
| remove solution verification (ablation) | UniDomain atomic domain learning · Atomic domain learning | plan validity | Hurt |
| remove predicate grouping (ablation) | UniDomain online planner · Evaluation tasks, especially Combination | task success rate | Hurt |
| remove task-relevant predicate and operator filtering (ablation) | UniDomain online planner · Evaluation tasks, especially BlockWorld | task success rate | Hurt |
| replace structured hierarchical fusion with direct LLM-based merging (ablation) | UniDomain domain fusion · Evaluation tasks | domain usability | Hurt |
Surprising findings
- ISR-LLM achieves the highest success among baselines but its optimality drops to match VLM-CoT at K=0, showing LLM-based planning hurts plan quality.
- A simple grayscale-intensity energy method for keyframe extraction outperforms SigLIP-2 embedding-based extraction in both speed and single-pass domain success.
Limitations
- PDDL 1.0 lacks temporal, numeric, and cost-sensitive planning support.
- Assumes full observability, ignoring occlusion and perceptual noise.
- Automatically retrieved atomic domains can be redundant, making meta-domain construction time-consuming.
Experimental setup
Datasets: DROID Agibot World
Benchmarks: UniDomain evaluation tasks (BlockWorld, Desktop, Kitchen, Combination; 100 tasks)
Metrics: success rate SPL optimality rate thinking time LLM calls
Baselines: Code-as-Policies ReAct VLM-CoT ISR-LLM VLM-PDDL BoN-iVML
Builds on: PDDL Fast Downward ISR-LLM BoN-iVML LLM+P
Closest papers by meaning
- PO-PDDL: Learning Symbolic POMDPs from Visual Demonstrations for Robot Planning Under Uncertainty (NeurIPS 2026 · similarity 0.91)
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.89)
- RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation (NeurIPS 2025 · similarity 0.89)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.88)
- Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks (ICLR 2024 · similarity 0.88)
- RoboMP$^2$: A Robotic Multimodal Perception-Planning Framework with Multimodal Large Language Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.