Research map / Reinforcement learning / Visual and representation RL
Ctrl-World: A Controllable Generative World Model for Robot Manipulation
Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, Chelsea Finn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-view joint prediction (wrist view) (ablation) | Ctrl-World · DROID val, wrist-view | FVD: 158.1 -> 127.1 | Improved |
| add frame-level action conditioning (ablation) | Ctrl-World · DROID val, wrist-view | FVD: 179.1 -> 127.1 | Improved |
| add frame-level action conditioning (ablation) | Ctrl-World · DROID val, third-view | FVD: 122.7 -> 97.4 | Improved |
| add pose-conditioned memory retrieval (ablation) | Ctrl-World · DROID val, third-view | FVD: 105.5 -> 97.4 | Improved |
| add pose-conditioned memory retrieval (ablation) | Ctrl-World · DROID val, wrist-view | FVD: 133.1 -> 127.1 | Improved |
| use multi-view prediction instead of single-view | Ctrl-World · DROID val, third-view | FVD: 127.5 -> 97.4 | Improved |
| use multi-view prediction instead of single-view | Ctrl-World · DROID val, third-view | FID: 23.47 -> 25.00 | Hurt |
| Ctrl-World-Single-View vs IRASim-Single-View | single-view world model · DROID val, third-view | PSNR: 21.36 vs 21.27 | Mixed |
| post-train pi0.5 on world-model synthetic successful trajectories | pi0.5-DROID · novel instructions and objects | success rate (%): 38.7 -> 83.4 | Improved |
| post-train on synthetic data | pi0.5-DROID · Towel folding with directions | success rate: 0.575 -> 0.80 | Improved |
| evaluate policies in world model instead of real world | Ctrl-World · Drawer and Close-laptop success rate | success rate: 0.80 -> 0.30 (Drawer, pi0.5); 0.70 -> 0.05 (Close-laptop, pi0.5) | Hurt |
Later papers that compared with this method
2 later accepted papers report Ctrl-World as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators (NeurIPS 2026 · better on 5 of 5 shared results)
- Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models (ICML 2026 · better on 3 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.90)
- FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model (ICLR 2025 · similarity 0.90)
- Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination (ICLR 2025 · similarity 0.90)
- HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation (ICLR 2025 · similarity 0.89)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.89)
- GENMANIP: LLM-driven Simulation for Generalizable Instruction-Following Manipulation (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
