Research map / Large language models / Language, safety and interpretability
Scaling Large Motion Models with Million-Level Human Motions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MotionLib, a 1.2M-clip web-video motion-text dataset with hierarchical captions, and trains Being-M0, a LLM-based large motion model using a lossless SMPL-D135 feature and a 2D lookup-free motion tokenizer (2D-LFQ).
Problem
Large motion models for text-to-motion generation are limited by scarce high-quality motion data and by lossy, small-codebook motion tokenizers; it is unclear whether scaling data and model size helps motion generation.
Approach
Scale motion data to over 1.2M clips extracted from web videos with hierarchical part- and body-level captions, and encode motions as single-channel 2D images quantized with a lookup-free (zero-dimensional-embedding, binary) quantizer that enables a 16384-entry codebook without collapse; use a lossless SMPL-based feature instead of H3D.
Key results
- On HumanML3D, Being-M0-LFQ with LLaMA2-13B reaches R@1 0.528 and R@3 0.820, with MMDist 2.875, the best among compared methods.
- Scaling data from 0.08M (Motion-X) to 1.2M (MotionLib) with LLaMA2-13B raises MotionLib-eval R@1 from 0.124 to 0.185 and lowers FID from 21.0 to 6.2.
- On out-of-distribution UNSEEN-90K, MotionLib-trained GPT-2 reaches R@1 0.098 vs 0.034 for HumanML3D and 0.051 for Motion-X.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase model size from LLaMA2-7B to LLaMA2-13B | Being-M0 · Motion-X-eval | R@1: 0.475 -> 0.491 | Helped |
| increase training data from 0.5M to 1.2M | Being-M0 with LLaMA2-13B · MotionLib-eval | R@1: 0.174 -> 0.185 | Helped |
| train on MotionLib-#11 rather than HumanML3D or Motion-X | GPT-2 motion model · UNSEEN-90K | R@1: 0.034 / 0.051 -> 0.098 | Helped |
| add motion instruction tuning (ablation) | Being-M0 · Motion-X | R@1: 0.471 -> 0.488 | Helped |
| use hierarchical rather than single-level descriptions (ablation) | Being-M0 · MotionLib-eval | R@1: 0.162 -> 0.166 | Helped |
| use SMPL-D135 rather than H3D-D263 features (ablation) | Being-M0 · MotionLib-eval | R@1: 0.514 -> 0.529 | No clear effect |
| replace VQ-VAE with 2D-LFQ (ablation) | motion tokenizer · MotionLib reconstruction | MPJPE: 123.6 -> 64.1 | Helped |
| use full-parameter fine-tuning rather than LoRA (ablation) | Being-M0 · MotionLib-eval | R@1: 0.157 -> 0.166 | Helped |
| fine-tune pretrained GPT-2 rather than train from scratch (ablation) | GPT-2 motion model · Motion-X-eval, 0.08M instructions | R@1: 0.461 -> 0.468 | Helped |
| remove description masking (ablation) | Being-M0 · HumanML3D | R@1: 0.388 -> 0.466 | Helped |
| use decoder-only rather than encoder-decoder architecture (ablation) | motion generation model · MotionLib-eval | R@1: 0.161 -> 0.166 | Helped |
| use 2D rather than 1D LFQ (ablation) | motion tokenizer · Motion-X reconstruction | MPJPE: 78.9 -> 54.1 | Helped |
Surprising findings
- LoRA fine-tuning performs far worse than full-parameter tuning for large motion models (R@1 0.157 vs 0.166 on MotionLib, FID 9.287 vs 6.936), attributed to the newly added motion tokens.
- The 2D-LFQ tokenizer underperforms RQ-VAE on the small HumanML3D reconstruction task (FID 0.092 vs 0.052), while outperforming it on larger Motion-X and MotionLib.
Limitations
- Motion-X and MotionLib evaluation uses a motion autoencoder retrained by the authors, so absolute metrics are not directly comparable to published numbers.
- Large motion models converge slowly (about 200 epochs), attributed to the limited representational capacity of the motion tokenizer.
- Much of the data is pseudo-labeled by LMMs (Gemini-1.5-Pro) and refined via RL-based physics policy, with some noisy motions down-weighted; data quality is validated only by GPT-4o scoring.
Experimental setup
Datasets: MotionLib HumanML3D Motion-X Kinetics-700 NTU-RGBD-120 BEDLAM GTA-Human AMASS
Benchmarks: HumanML3D Motion-X-eval MotionLib-eval UNSEEN-90K
Metrics: R-Precision FID MMDist MPJPE
Baselines: T2M-GPT MoMask MLD MotionDiffuse ReMoDiffuse MotionGPT MotionGPT-v2 MotionLLM AvatarGPT RQ-VAE H2VQ
Builds on: T2M-GPT MotionGPT LFQ (lookup-free quantization) HumanML3D feature WHAM
Closest papers by meaning
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.88)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.88)
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.88)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.88)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.88)
- How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.