Research map / 3D vision / Face and shape
Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates SMPL body shape parameters and shape-aware human motions from text by combining a shape-conditioned FSQ-VAE with a fine-tuned T5 that predicts motion tokens and a shape token.
Problem
Existing text-to-motion methods canonicalize motions to a homogenized body shape, losing the correlation between body shape and motion dynamics; shape-aware motion generation from natural language is underexplored.
Approach
Quantize shape-normalized motion with an FSQ-VAE, then de-quantize and condition decoding on continuous shape parameters; a T5 language model jointly predicts discrete motion tokens and a continuous shape parameter read out from a special [BETA] token embedding.
Key results
- ShapeMove achieves FID 0.125 on HumanML3D-based shape-aware evaluation versus 0.151 for T2M-GPT in the SA-VAE quantizer comparison, with bone length difference of 45.88 mm vs 83.42 mm for T2M-GPT.
- In Table 1, ShapeMove gets the best Penetrate (0.0268 cm) among compared methods, and best Skate and Bone Length metrics among shape-capable baselines.
- Human perceptual study (45 respondents, Mechanical Turk) shows ShapeMove preferred roughly 12% to 38% over baselines across three metrics; predicted shape attributes differ from ground truth by about 1 cm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use shape-conditioning instead of no shape-conditioning (ablation) | ShapeMove · HumanML3D reconstruction | Bone Length Diff. (mm): 99.18 -> 66.41 | Helped |
| Add bone length loss (ablation) | ShapeMove with shape-conditioning · HumanML3D reconstruction | Bone Length Diff. (mm): 66.41 -> 45.11 | Helped |
| Add float loss (ablation) | ShapeMove with shape-conditioning and bone length loss · HumanML3D reconstruction | Float (cm): 0.480 -> 0.255 | Helped |
| Add foot skate loss (ablation) | ShapeMove with shape-conditioning, bone length loss, and float loss · HumanML3D reconstruction | Skate (%): 7.90 -> 6.14 | Helped |
| Use ShapeMove rather than the strongest listed baseline | HumanML3D text-to-motion · HumanML3D test | Penetrate (cm): 0.0939 -> 0.0268 | Helped |
| Use SA-VAE rather than T2M-GPT quantization | Motion reconstruction · HumanML3D reconstruction | Bone Length Diff. (mm): 83.42 -> 45.88 | Helped |
Surprising findings
- Adding the bone length loss reduced bone length difference sharply but the float loss increased FID in the ablation, showing trade-offs among physical losses.
Limitations
- Shape descriptions must follow a fixed template, limiting natural-language flexibility.
- Relies on HumanML3D, which has limited subject diversity (449 subjects), supplemented by synthetic shapes.
- Baselines T2M and TM2T cannot take shape descriptions and were evaluated with motion-only input.
Experimental setup
Datasets: HumanML3D AMASS HumanAct12 Shapy A2S synthetic shapes
Benchmarks: HumanML3D
Metrics: FID R-Precision MM-Dist Diversity Penetrate Float Skate Bone Length Variances Bone Length Diff Jitter Diff
Baselines: T2M TM2T MLD MotionDiffuse MDM T2M-GPT MotionGPT
Builds on: T2M-GPT MotionGPT FSQ T5 Shapy
Closest papers by meaning
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.87)
- HumanTOMATO: Text-aligned Whole-body Motion Generation (ICML 2024 · similarity 0.87)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.87)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.87)
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.87)
- Controllable Dynamic 3D Shape Generation via 3D Trajectories and Text (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.