Research map / Video / Action recognition
CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learning
Jiange Yang, Yansong Shi, Haoyi Zhu, Mingyu Liu, Kaijing Ma, Yating Wang, Gangshan Wu, Tong He, Limin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add early temporal difference and temporal contrastive learning to continuous latent motion (ablation) | CoMo · LIBERO (10 demos/task) | success rate (%): 75.2 -> 80.1 | Improved |
| add early temporal difference only (ablation) | continuous latent motion · LIBERO | success rate (%): 75.2 -> 76.9; MSE 1.6256 -> 1.5209 | Improved |
| add temporal contrastive learning only (ablation) | continuous latent motion · LIBERO | S-PCFC: 0.9268 -> 0.6229 | Improved |
| remove vector quantization from latent motion (naive continuous) (ablation) | latent motion baseline · LIBERO | S-PCFC: S-PCFC 0.9268 (near 1.0), success rate 75.2 vs discrete 75.9 | Hurt |
| use continuous instead of discrete latent motion (with Td and Tcl) (ablation) | CoMo · LIBERO | success rate (%): 75.9 -> 80.1 | Improved |
| use continuous instead of discrete latent motion (with Td and Tcl) (ablation) | auto-regressive policy · CALVIN ABC->D | average length: 2.255 -> 3.070 | Improved |
| add Tcl to continuous baseline in auto-regressive policy (ablation) | Con. · CALVIN ABC->D | average length: 2.797 -> 2.925 | Improved |
| add Td on top of Tcl (ablation) | Con.+Tcl · CALVIN ABC->D | average length: 2.925 -> 3.070 | Improved |
| expand latent motion dimension from 128 to 512 (ablation) | CoMo · CALVIN ABC->D | average length: 3.070 -> 3.247 | Improved |
| expand latent motion dimension with Td only (no Tcl) (ablation) | Con.+Td · LIBERO | action prediction MSE | Hurt |
| use future frame features as latent motion (ablation) | latent motion baseline · LIBERO | success rate (%): MSE 2.1442 vs Dis. 5.6743, but success rate 73.8 vs 75.9 | Hurt |
| add CoMo pseudo-labelled video to diffusion policy | diffusion policy · LIBERO | success rate (%): 70.4 -> 80.1 | Improved |
Closest papers by meaning
- ViPRA: Video Prediction for Robot Actions (ICLR 2026 · similarity 0.91)
- Learning a Unified Latent Action Space from Videos with Action-centric Cycle Consistency (CVPR 2026 · similarity 0.90)
- Latent Action Pretraining from Videos (ICLR 2025 · similarity 0.89)
- Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training (NeurIPS 2024 · similarity 0.88)
- DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control (NeurIPS 2026 · similarity 0.88)
- RotVLA: Rotational Latent Action for Vision-Language-Action Model (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
