Research map / Video / Video-language understanding
Compositional Foundation Models for Hierarchical Planning
Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi S. Jaakkola, Joshua B. Tenenbaum, Leslie Pack Kaelbling, Akash Srivastava, Pulkit Agrawal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compose task, visual, and action planning hierarchically | HiP · Paint-block, Object-arrange, and Kitchen-tasks (seen and unseen) | task/subtask completion rate: Paint-block: 74.3\pm1.9 (seen), 72.8\pm1.7 (unseen); Object-arrange: 75\pm2.8, 75.4\pm2.6; Kitchen-tasks: 85.8\pm 9.4, 8 | Improved |
| use oracle subgoals instead of predicted subgoals (ablation) | HiP · Paint-block, Object-arrange, and Kitchen-tasks | task/subtask completion rate: Paint-block: 74.3\pm1.9 -> \bf{81.2}\pm1.8 (seen); 72.8\pm1.7 -> \bf{79.6}\pm1.9 (unseen) | Improved |
| remove task-plan refinement for visual grounding (ablation) | HiP · Paint-block | task completion rate | Hurt |
| remove both task-plan and visual-plan refinement (ablation) | HiP · Paint-block | task completion rate | Hurt |
| remove visual-plan refinement (ablation) | HiP · Paint-block | task completion rate | Hurt |
| pretrain video diffusion on Ego4D (ablation) | HiP video planner · Long-horizon planning | success rate | Improved |
| pretrain video diffusion on Ego4D (ablation) | HiP video planner · Long-horizon planning | FVD | Improved |
| initialize inverse dynamics with VC-1 weights instead of random initialization (ablation) | HiP inverse dynamics · Paint-block and Object-arrange | mean squared error: 1K labeled robotic trajectories vs 10K labeled robotic trajectories | Improved |
| initialize inverse dynamics with VC-1 weights instead of random initialization (ablation) | HiP inverse dynamics · Kitchen-tasks | mean squared error: 3.5k labeled robotic trajectories vs 10K labeled robotic trajectories | Improved |
| replace the learned subgoal classifier with frozen MiniGPT-4 (ablation) | HiP task planner · Paint-block | subgoal prediction accuracy | No clear effect |
| replace the learned subgoal classifier with frozen MiniGPT-4 (ablation) | HiP task planner · Object-arrange | subgoal prediction accuracy | Hurt |
Closest papers by meaning
- Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts (CVPR 2024 · similarity 0.88)
- FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model (ICLR 2025 · similarity 0.88)
- Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification (ICML 2025 · similarity 0.88)
- VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models (NeurIPS 2025 · similarity 0.88)
- SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution (CVPR 2024 · similarity 0.88)
- Video Language Planning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
