Research map / Video / Video generation
Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, Jinwei Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize from pretrained video model instead of random weights (ablation) | Cosmos Policy · LIBERO (average) | success rate (%): 94.6 -> 98.5 | Improved |
| initialize from pretrained video model instead of random weights (ablation) | Cosmos Policy · ALOHA fold shirt | score: 80.8 -> 18.7 points higher full model | Improved |
| add auxiliary future-state and value prediction targets (ablation) | Cosmos Policy · LIBERO (average) | success rate (%): 97.0 -> 98.5 | Improved |
| add value function training samples (ablation) | Cosmos Policy · RoboCasa (average) | success rate (%): 66.6 -> 67.1 | Improved |
| add world model training samples (ablation) | Cosmos Policy · RoboCasa (average) | success rate (%): 64.0 -> 66.6 | Improved |
| add model-based planning with V(s') using refined world model (ablation) | Cosmos Policy · ALOHA two hardest tasks | average score: +12.5 points | Improved |
| plan with state value V(s') via world model instead of model-free Q(s,a) (ablation) | Cosmos Policy planning · ALOHA two hardest tasks | score | Improved |
| fine-tune pretrained video model into policy instead of fine-tuned VLAs | LIBERO · LIBERO (average) | success rate (%): 97.4 -> 98.5 (vs CogVLA) | Improved |
| use Cosmos Policy instead of pi_0.5 on out-of-distribution ALOHA tests | ALOHA · Real-world ALOHA (OOD only) | average score: 92.5 -> 89.3 | Hurt |
Later papers that compared with this method
3 later accepted papers report Cosmos-Policy as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- SwiftWA: An Efficient Action-Centered World-Action Model (ECCV 2026 · better on 4 of 4 shared results)
- Learning Transferable Dynamics Priors from Action to World Modeling (ECCV 2026 · better on 2 of 2 shared results)
- Action Images: End-to-End Policy Learning via Multiview Video Generation (NeurIPS 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- GraphMimic: Graph-to-Graphs Generative Modeling from Videos for Policy Learning (CVPR 2025 · similarity 0.89)
- VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation (NeurIPS 2024 · similarity 0.89)
- One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation (ICML 2025 · similarity 0.89)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.88)
- VideoVLA: Video Generators Can Be Generalizable Robot Manipulators (NeurIPS 2025 · similarity 0.88)
- Two-Steps Diffusion Policy for Robotic Manipulation via Genetic Denoising (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
