Research map / Reinforcement learning / Visual and representation RL
Hierarchical World Models with Implicit Dynamics
Gaoyue Zhou, Yvonne Wu, Zichen Cui, Nicolas Ballas, Mahmoud Assran, Lerrel Pinto, Yann LeCun
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Implicit-HWM, a hierarchical world model for embodied agents that plans with implicit dynamics to handle long-horizon tasks.
Problem
Long-horizon planning with world models for embodied agents suffers from cumulative errors and search complexity, and existing hierarchical methods rely on rigid macro-actions or latent skills that yield brittle representations and limited fine-grained control.
Approach
A hierarchical world model using implicit dynamics as an alternative to rigid macro-actions or latent skills, intended to support fine-grained control while enabling test-time adaptation via planning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add hierarchy (high-level diffusion subgoal model) over single-level WM with MPC | Single-level WM · PushT (random goals) | success rate: 0.21 -> 0.77 | Improved |
| Triple replanning budget (45 replans) for single-level WM to match inference time (ablation) | SL-WM · PushT (random goals) | success rate: 0.21 -> 0.28 | No clear effect |
| Use diffusion noise as implicit action instead of macro-actions (vs HWM) | Hierarchical WM · PushT-Aug (random goals) | success rate: 0.16 -> 1.00 | Improved |
| Use hierarchy over DINO-WM in vision-based PushT, long horizon | DINO-WM low level · Vision PushT (d = 75) | success rate: 0.00 -> 0.50 | Improved |
| Use Implicit-HWM instead of HWM at short horizon | HWM · Vision PushT (d = 25) | success rate: 0.96 -> 0.90 | Hurt |
| Add non-zero latent noise regularization (alpha > 0) (ablation) | Implicit-HWM planner · PushT success rate | success rate | Improved |
| Increase number of CEM samples (ablation) | Implicit-HWM · PushT | success rate | Improved |
| Use very small subgoal radius (Delta = 1, 25 subgoals vs Delta = 5, 5 subgoals) (ablation) | Implicit-HWM · PushT-Aug (random goals) | success rate: 0.96 -> 0.06 | Hurt |
| Increase number of subgoals to 25 at Delta = 5 (ablation) | Implicit-HWM · PushT-Aug (random goals) | success rate: 0.96 -> 0.38 | Hurt |
| Add data augmentation to training data (PushT-Aug) | H-Diffuser · PushT (random goals) | success rate: 0.01 -> 0.27 | Mixed |
| Add replanning at every action | Diffuser · PushT | success rate | No clear effect |
| Hierarchy vs reactive baseline on in-distribution fixed goal | Implicit-HWM vs GCBC · PushT (fixed goal) | success rate: 1.00 vs 1.00 | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- HIQL — HIQL: Offline Goal-Conditioned RL with Latent States as Actions (NeurIPS 2023) · better on 4 of 4 shared results
- DINO-WM — DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning (ICML 2025) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Predictive but Not Plannable: RC-aux for Latent World Models (NeurIPS 2026 · similarity 0.89)
- DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning (ICML 2025 · similarity 0.88)
- World Models as Reference Trajectories for Rapid Motor Adaptation (NeurIPS 2025 · similarity 0.88)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.87)
- WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents (NeurIPS 2025 · similarity 0.87)
- Sparse Planning in Visual World Models via Cost Gradients (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
