Research map / Video / Video generation
MMM: Generative Masked Motion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MMM, a text-to-motion model that tokenizes 3D motion with a factorized VQ-VAE and generates it via a CLIP-conditioned masked transformer with parallel iterative decoding, enabling fast, editable motion synthesis.
Problem
Text-to-motion generation suffers from a trade-off between inference speed, fidelity, and motion editability: diffusion models are slow, and autoregressive models are slow and hard to edit.
Approach
Applies BERT-style masked token modeling to discrete motion tokens (from a large factorized-codebook VQ-VAE) conditioned on CLIP word and sentence embeddings, so parallel iterative decoding is fast and placing [MASK] tokens enables in-betweening, body-part editing, and long-sequence generation without extra training.
Key results
- Achieves FID 0.080 on HumanML3D and 0.429 diversity... FID 0.590 vs. 0.621 for next best on KIT-ML is not best; on HumanML3D MMM reaches FID 0.080 versus 0.112 for AttT2M.
- Average inference time per sentence of 0.081 s on a single RTX A5000, at least 2x faster than autoregressive models and about two orders of magnitude faster than motion-space diffusion (MDM 28.112 s).
- Generates a 10.873-minute motion sequence in 1.658 seconds via transition token infilling without extra training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MMM instead of AttT2M | Text-to-motion generation · HumanML3D test set | FID: 0.112±.006 -> 0.080±.003 | Helped |
| use MMM instead of AttT2M | Text-to-motion generation · HumanML3D test set | R-Precision Top-1: 0.499±.003 -> 0.504±.003 | Helped |
| use MMM instead of MLD | Text-to-motion generation · KIT-ML test set | FID: 0.404±.027 -> 0.429±.019 | Hurt |
| use MMM instead of Fg-T2M | Text-to-motion generation · KIT-ML test set | R-Precision Top-1: 0.418±.005 -> 0.381±.005 | Hurt |
| use training masking ratio Uniform .3-1 instead of Uniform 0-1 (ablation) | MMM · HumanML3D | FID: 0.097 -> 0.089 | Helped |
| use training masking ratio Uniform .5-1 instead of Uniform 0-1 (ablation) | MMM · HumanML3D | FID: 0.097 -> 0.089 | Helped |
| use training masking ratio Uniform .7-1 instead of Uniform 0-1 (ablation) | MMM · HumanML3D | FID: 0.097 -> 0.098 | Hurt |
| use 10 inference iterations instead of 5 (ablation) | MMM · HumanML3D | FID: 0.169 -> 0.089 | Helped |
| use 10 inference iterations instead of 5 (ablation) | MMM · HumanML3D | AITS (seconds): 0.043 -> 0.081 | Hurt |
| use 15 inference iterations instead of 10 (ablation) | MMM · HumanML3D | FID: 0.089 -> 0.091 | Hurt |
| use 20 inference iterations instead of 10 (ablation) | MMM · HumanML3D | FID: 0.089 -> 0.096 | Hurt |
| use codebook 8192 × 32 instead of 512 × 512 (ablation) | MMM · HumanML3D | FID: 0.108 -> 0.075 | Helped |
Surprising findings
- Motion in-betweening and long-sequence transitions work without any training on transition data, using the same masked model.
Limitations
- Trained only on short motion datasets (max ~10 s), so long sequences rely on transition generation rather than learned long-range data.
- KIT-ML results are weaker on R-Precision and diversity relative to some baselines.
- Upper-body editing requires separately trained upper and lower body tokenizers.
Experimental setup
Datasets: HumanML3D KIT-ML
Benchmarks: HumanML3D KIT-ML
Metrics: R-Precision FID MM-Dist Diversity MultiModality AITS
Baselines: MDM MotionDiffuse MLD T2M-GPT AttT2M Fg-T2M M2DM TEMOS TM2T T2M
Builds on: VQ-VAE MaskGIT T2M-GPT CLIP MDM
Closest papers by meaning
- MoMask: Generative Masked Modeling of 3D Human Motions (CVPR 2024 · similarity 0.92)
- MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformer (ICLR 2025 · similarity 0.90)
- SnapMoGen: Human Motion Generation from Expressive Texts (NeurIPS 2025 · similarity 0.90)
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.90)
- SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Prediction (CVPR 2025 · similarity 0.90)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.