Research map / Reinforcement learning / Visual and representation RL
Lifting Ego World Models for Planning and Control
Alex Wang, Trevor Darrell, Pavel Izmailov, Yutong Bai, Amir Bar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add waypoint conditioning (ablation) | NoMaD-like policy with pose context · Nymeria policy validation | goal-conditioned all-joint MJE (m): 0.323 -> 0.243 | Improved |
| add pose context (ablation) | NoMaD-like policy · Nymeria policy validation | goal-conditioned all-joint MJE (m): 0.367 -> 0.323 | Improved |
| add architecture changes (DINOv3, larger UNet) (ablation) | NoMaD-like policy · Nymeria policy validation | goal-conditioned all-joint MJE (m): 0.392 -> 0.367 | Improved |
| add goal observation conditioning to NoMaD baseline (ablation) | NoMaD-like policy · Nymeria policy validation | MJE reduction: 2.1cm -> 1.3cm | No clear effect |
| remove goal masking in policy training (ablation) | waypoint-conditioned policy · Nymeria policy validation | goal-conditioned all-joint MJE (m): 0.243 -> 0.226 | Mixed |
| add depth to waypoints (3D conditioning) (ablation) | waypoint-conditioned policy · Nymeria policy validation | goal-conditioned all-joint MJE (m): 0.243 -> 0.226 | No clear effect |
| plan with Lifted CEM (2D waypoints) instead of PEVA CEM in joint space | PEVA world model · Nymeria planning (128 tasks, CEM) | all-joint MJE (m): 0.616 -> 0.374 | Improved |
| search 3D waypoints instead of 2D waypoints in CEM (ablation) | Lifted CEM · Nymeria planning (128 tasks, CEM) | all-joint MJE (m): 0.374 -> 0.420 | Hurt |
| plan with Lifted CEM instead of goal-image policies (NoMaD, HDP) | PEVA world model · Nymeria planning (128 tasks, CEM) | all-joint MJE (m): 0.512 -> 0.374 | Improved |
| plan with lifted world model on unseen environments | Lifted CEM · Nymeria planning, unseen environments | all-joint MJE (m): 0.553 -> 0.362 | Improved |
| lift DexWM with waypoints for planning | DexWM · EgoDex planning with DexWM (256 tasks) | all-joint MJE (cm): 13.81 -> 7.98 | Improved |
| add waypoint masking in policy training (ablation) | Lifted CEM · Nymeria planning (64 tasks) | leaf MJE (m): 0.399 -> 0.388 | Improved |
Closest papers by meaning
- World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.87)
- SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators (NeurIPS 2026 · similarity 0.87)
- Beyond World-Frame Action Heads: Motion-Centric Action Frames for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.86)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.86)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.86)
- DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
