Research map / Video / Video generation
MotionCtrl: A Real-time Controllable Vision-Language-Motion Model
Bin Cao, Sipeng Zheng, Ye Wang, Lujie Xia, Qianshan Wei, Qin Jin, Jing Liu, Zongqing Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace VQ with part-aware residual quantization (PRQ) | MotionCtrl · HumanML3D (text-to-motion) | FID: 0.141 to 0.056 | Improved |
| share joints across body parts in PRQ (ablation) | MotionCtrl PRQ4 · HuMo-I2PM | FID: 2.471 to 1.831 | Improved |
| share joints across body parts in PRQ (ablation) | PRQ4 tokenizer · HumanML3D reconstruction | FID: 0.042 -> 0.007 | Improved |
| use part-level descriptions in training (ablation) | MotionCtrl-PRQ4 · HuMo-I2PM | FID: 4.281 -> 1.831 | Improved |
| use part-level descriptions in training (ablation) | MotionCtrl-RQ6 · HuMo-I2PM | FID: 4.025 -> 3.752 | No clear effect |
| increase PRQ quantization layers from 4 to 6 (ablation) | MotionCtrl · HuMo-I2PM | FID: 1.831 -> 2.357 | Hurt |
| increase PRQ quantization layers from 4 to 6 (ablation) | PRQ tokenizer · HumanML3D reconstruction | FID: 0.007 -> 0.004 | Improved |
| add vision-text-motion alignment stage (ablation) | MotionCtrl · HuMo-T2M | FID: 7.053 -> 5.791 | Improved |
| add MPI task to multi-task training (ablation) | MotionCtrl · HuMo-T2M | FID: 6.582 -> 6.052 | Improved |
| add I2PM and I2LM tasks to multi-task training (ablation) | MotionCtrl · HuMo-T2M | FID: 6.052 -> 5.791 | Improved |
| add I2M task to T2M training (ablation) | MotionCtrl · HuMo-T2M | FID: 7.152 -> 6.582 | Improved |
| scale training data from HumanML3D to HuMo100M | MotionCtrl · HuMo-unseen I2U | FID: 65.04 -> 8.65 | Improved |
Closest papers by meaning
- Multimodal Context-Aware Human Motion Generation with Language, Vision, and Object (NeurIPS 2026 · similarity 0.90)
- MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators (AAAI 2024 · similarity 0.89)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.88)
- HMVLM:Human Motion-Vision-Language Model via MoE LoRA (NeurIPS 2025 · similarity 0.88)
- HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (CVPR 2025 · similarity 0.88)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
