Research map / Architectures and efficiency / RNNs and normalization
ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ScaMo, a scalable text-to-motion system (Motion FSQ-VAE tokenizer plus a text-prefix autoregressive transformer trained on the new MotionUnion dataset) and fits scaling laws for normalized test loss, vocabulary/non-vocabulary parameters and data versus FLOPs.
Problem
Scaling laws have not been verified for autoregressive text-driven human motion generation, which is limited by small motion datasets, codebook collapse when scaling motion tokenizer vocabularies, and architectures that do not scale well with model size.
Approach
Replace VQ with finite scalar quantization (FSQ) so the motion vocabulary can scale without codebook collapse, and use a text-prefix autoregressive transformer with frozen T5-XL word embeddings (bidirectional on text, causal on motion), then fit Chinchilla-style power laws with vocabulary parameters separated from non-vocabulary parameters.
Key results
- FSQ achieves lower reconstruction L1 and MPJPE than VQ at large codebook sizes and keeps codebook utilization near the theoretical maximum, while VQ utilization drops as codebook grows.
- Text-prefix T5-XL model (343M) improves FID from 0.226 (CLIP, no prefix) to 0.104 and Top-1 R-Precision from 0.402 to 0.510 on HumanML3D.
- Fitted logarithmic law L_u = -1.062 log10(C) + 13.839; predicted optimal 3B model, vocabulary 2^16 and 1e7.5 data for compute 1e18, with the trained model aligning with the predicted loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a T5-XL text encoder with text-prefix design instead of CLIP without prefix (ablation) | ScaMo, 343M · HumanML3D | FID: 0.226 -> 0.104 | Helped |
| Use a T5-XL text encoder with text-prefix design instead of CLIP without prefix (ablation) | ScaMo, 343M · HumanML3D | Matching Score: 3.422 -> 3.021 | Helped |
| Use a T5-XL text encoder with text-prefix design instead of CLIP without prefix (ablation) | ScaMo, 343M · HumanML3D | Top1 R-Precision: 0.402 -> 0.510 | Helped |
| Replace vanilla VQ with FSQ (ablation) | Motion tokenizer · HumanML3D and MotionUnion | Reconstruction loss | Helped |
| Replace vanilla VQ with FSQ (ablation) | Motion tokenizer · HumanML3D and MotionUnion | MPJPE | Helped |
| Replace vanilla VQ with FSQ (ablation) | Motion tokenizer · HumanML3D and MotionUnion | Codebook utilization | Helped |
| Replace vanilla VQ with FSQ (ablation) | Motion tokenizer · HumanML3D and MotionUnion | Exponential entropy | Helped |
Surprising findings
- FID is biased toward the motion domain and fails to distinguish between models in scaling experiments, so the authors use normalized test loss instead.
- Directly extending the language vocabulary with motion tokens was found not effective enough for text-driven motion generation.
Limitations
- Scaling experiments are limited to models from 44M to 3B parameters and vocabulary sizes up to 2^16.
- Grouped or residual FSQ extensions are left as future work.
- The MotionUnion dataset is not released, limiting reproducibility.
Experimental setup
Datasets: MotionUnion HumanML3D Motion-X CombatMotion 100 Style EgoBody Idea400 HAA500
Benchmarks: HumanML3D MotionUnion
Metrics: FID R-precision matching score normalized test loss L1 loss MPJPE codebook usage exponential entropy
Baselines: T2M-GPT VQ-VAE (vanilla codebook) CLIP text encoder variant
Builds on: T2M-GPT Chinchilla scaling law Finite Scalar Quantization Tao et al. normalized loss scaling
Closest papers by meaning
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.88)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.87)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.87)
- SAMPO: Scale-wise Autoregression with Motion Prompt for Generative World Models (NeurIPS 2025 · similarity 0.87)
- Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens (ICLR 2025 · similarity 0.87)
- Fundamental Limits of Visual Autoregressive Transformers: Universal Approximation Abilities (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.