Research map / Video / Video generation
AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AMD, which uses a fine-tuned ChatGPT-3.5 to decompose text into anatomical scripts and fuses a canonical diffusion branch with a retrieval-based reference-motion diffusion branch for text-to-motion generation on complex text.
Problem
Text-to-human-motion diffusion models struggle with complex or long text descriptions and rare actions, because a single text encoder compresses text to a fixed-length vector.
Approach
Decompose complex text into simple anatomical scripts with a fine-tuned LLM, retrieve reference motions for those scripts, and run two diffusion branches (full text and scripts+references) whose outputs are blended by a Bernoulli-style fusion controlled by λ.
Key results
- On SLCD1, AMD achieves FID 0.116 vs 0.603 for the best baseline T2M-GPT and 0.830 for MotionDiffuse, with R-Precision 0.673.
- On SLCD2, AMD achieves FID 0.146 but R-Precision 0.667, lower than MotionDiffuse's 0.742.
- On HumanML3D, AMD reports FID 0.204 and R-Precision 0.657, while T2M-GPT reports R-Precision 0.775 and FID 0.116, so AMD is not best on all metrics; on KIT performance was below expectations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text decomposition (ablation) | AMD · HumanML3D | FID: 0.65±.07 -> 0.20±.03 | Helped |
| add text decomposition (ablation) | AMD · HumanML3D | Multi-Modality: 1.81±.11 -> 1.36±.09 | Hurt |
| add text decomposition and reference action information (ablation) | AMD · HumanML3D | FID: 1.39±.08 -> 0.20±.03 | Helped |
| add reference action information to text decomposition (ablation) | AMD · HumanML3D | Multi-Modality: 2.88±.10 -> 1.36±.09 | Hurt |
| use AMD instead of MotionDiffuse | text-to-motion generation · SLCD1 | FID: 0.830±.043 -> 0.116±.011 | Helped |
| use AMD instead of MotionDiffuse | text-to-motion generation · SLCD2 | R Precision (top 3): 0.742±.004 -> 0.667±.005 | Hurt |
| use AMD instead of ReMoDiffuse | text-to-motion generation · HumanML3D | FID: 0.103±.004 -> 0.204±.031 | Hurt |
| use AMD instead of ReMoDiffuse | text-to-motion generation · KIT | R-Precision: 0.765±.055 -> 0.401±.005 | Hurt |
Surprising findings
- On the KIT dataset, performance was below expectations, which the authors attribute to its short text descriptions.
- Text decomposition lowers Multimodality (breaking sentences into discrete steps constrains per-step generation).
Limitations
- Performance depends on the breadth of the retrieval motion database; small databases give low-quality references.
- Worse than some baselines on R-Precision and on KIT, and on the standard HumanML3D benchmark.
Experimental setup
Datasets: HumanML3D KIT-ML SLCD1 SLCD2
Benchmarks: HumanML3D KIT-ML SLCD1 SLCD2
Metrics: R-Precision FID Multimodal Distance Diversity Multimodality
Baselines: MDM TM2T MotionDiffuse T2M-GPT Guo et al. (2022a) ReMoDiffuse DiffKFC MMA MDF
Builds on: MDM ReMoDiffuse DDPM classifier-free guidance
Closest papers by meaning
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.91)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.90)
- SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing (CVPR 2025 · similarity 0.88)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.88)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.87)
- How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.