Research map / Architectures and efficiency / Attention and transformers
MoMask: Generative Masked Modeling of 3D Human Motions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
MoMask generates text-driven 3D human motion by tokenizing motion with a residual VQ-VAE and using two bidirectional masked transformers (base-layer masked prediction and residual-layer prediction), achieving state-of-the-art FID on HumanML3D and KIT-ML.
Problem
Text-to-motion generation suffers from quantization errors in single-layer VQ tokenizers and from unidirectional autoregressive decoding, while diffusion alternatives are slow.
Approach
Represent motion as multi-layer residual VQ tokens and generate them with a BERT-style masked transformer for the base layer (iterative parallel decoding with cosine mask schedule) followed by a residual transformer that predicts each subsequent residual layer.
Key results
- MoMask achieves FID 0.045 on HumanML3D (vs 0.141 for T2M-GPT) and 0.204 on KIT-ML (vs 0.514 for T2M-GPT) in Table 1.
- MoMask generates a full layered token set within about 10-18 inference iterations, faster than many diffusion and autoregressive baselines per Figure 5(a).
- In a user study with 42 AMT users, MoMask is preferred over MDM, MLD, and T2M-GPT most of the time; in temporal inpainting, 68% of MoMask results are preferred over MDM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MoMask instead of ReMoDiffuse | text-to-motion generation · HumanML3D | FID: 0.103±.004 -> 0.045±.002 | Helped |
| use MoMask instead of ReMoDiffuse | text-to-motion generation · KIT-ML | FID: 0.155±.006 -> 0.204±.011 | Hurt |
| use MoMask instead of ReMoDiffuse | text-to-motion generation · HumanML3D | R-Precision Top 1: 0.510±.005 -> 0.521±.002 | Helped |
| use MoMask instead of ReMoDiffuse | text-to-motion generation · KIT-ML | R-Precision Top 1: 0.427±.014 -> 0.433±.007 | Helped |
| add residual quantization (ablation) | MoMask · HumanML3D | FID: 0.093±.004 -> 0.045±.002 | Helped |
| add quantization dropout (ablation) | MoMask · HumanML3D | FID: 0.091±.003 -> 0.045±.002 | Helped |
| add replacing-and-remasking (ablation) | MoMask · HumanML3D | FID: 0.063±.003 -> 0.045±.002 | Helped |
| set residual quantization layers V to 0 (ablation) | MoMask · HumanML3D | FID: 0.045±.002 -> 0.093±.004 | Hurt |
| set residual quantization layers V to 1 (ablation) | MoMask · HumanML3D | FID: 0.045±.002 -> 0.073±.003 | Hurt |
| set residual quantization layers V to 2 (ablation) | MoMask · HumanML3D | FID: 0.045±.002 -> 0.072±.003 | Hurt |
| set residual quantization layers V to 3 (ablation) | MoMask · HumanML3D | FID: 0.045±.002 -> 0.064±.003 | Hurt |
| set residual quantization layers V to 4 (ablation) | MoMask · HumanML3D | FID: 0.045±.002 -> 0.069±.003 | Hurt |
Surprising findings
- Performance on KIT-ML degrades when using more than 5 residual layers, indicating the R-Transformer burden outweighs reconstruction gains.
- Increasing quantization dropout beyond 0.2 gives marginal or negative gains.
Limitations
- KIT-ML results are behind ReMoDiffuse, which uses retrieval from a large database.
- Multimodality is treated as a secondary metric and is not a strong point of the method.
- Evaluation relies on automatic metrics plus a crowd-sourced user study with limited sample sizes.
Experimental setup
Datasets: HumanML3D KIT-ML AMASS HumanAct12
Benchmarks: HumanML3D KIT-ML
Metrics: FID R-Precision MultiModal Distance Multimodality MPJPE
Baselines: TM2T T2M MDM MLD MotionDiffuse T2M-GPT ReMoDiffuse MotionGPT M2DM
Builds on: BERT MaskGIT T2M-GPT VQ-VAE residual quantization
Closest papers by meaning
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.92)
- SnapMoGen: Human Motion Generation from Expressive Texts (NeurIPS 2025 · similarity 0.88)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.87)
- MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformer (ICLR 2025 · similarity 0.87)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.87)
- Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.