Research map / Video / Video generation
Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes B2A-HDM, a hierarchical text-to-motion diffusion model that pairs a basic latent diffusion model (low-dim latent) for modality-consistent coarse denoising with an advanced high-dim latent diffusion model using multiple denoisers for detail-enhancing denoising.
Problem
Text-to-motion diffusion models either operate on raw motion data or a low-dimensional latent space, causing modality inconsistency or detail-scarce motion; higher-dimensional latents hurt text-motion consistency.
Approach
Split the reverse diffusion into sub-processes: a low-dimensional basic diffusion model produces early denoising steps for text consistency, then its output is re-noised into a high-dimensional latent space where an advanced diffusion model with two time-interval-specific denoisers adds motion details.
Key results
- On HumanML3D, B2A-HDM reaches FID 0.084 and Top-1 R-Precision 0.511, matching real-motion R-Precision and beating T2M-GPT's FID of 0.116.
- On KIT-ML, B2A-HDM attains FID 0.367 and Top-1 R-Precision 0.436, the best on both metrics among compared methods.
- In a human evaluation with 19 volunteers, B2A-HDM was selected as the best motion in 47.0% of assignments, versus 25.6% for MLD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine a 4×256 low-dimensional BDM with an 8×256 high-dimensional ADM (ablation) | B2A-HDM · HumanML3D | FID: 0.171 -> 0.084 | Helped |
| Combine a 4×256 low-dimensional BDM with an 8×256 high-dimensional ADM (ablation) | B2A-HDM · HumanML3D | R-Precision Top-1: 0.481 -> 0.511 | Helped |
| Increase the advanced diffusion rate from 25% to 95% (ablation) | B2A-HDM · HumanML3D | FID: 0.242 -> 0.084 | Helped |
| Increase the advanced diffusion rate from 25% to 95% (ablation) | B2A-HDM · HumanML3D | R-Precision Top-1: 0.509 -> 0.511 | No clear effect |
| Use three advanced denoisers instead of two (ablation) | B2A-HDM · HumanML3D | FID: 0.084 -> 0.130 | Hurt |
| Use four advanced denoisers instead of two (ablation) | B2A-HDM · HumanML3D | FID: 0.084 -> 0.139 | Hurt |
| Use one advanced denoiser instead of two (ablation) | B2A-HDM · HumanML3D | FID: 0.084 -> 0.315 | Hurt |
| Reduce the high-dimensional latent width from 8×256 to 4×256 while retaining a 4×256 low-dimensional space (ablation) | B2A-HDM · HumanML3D | FID: 0.084 -> null | No clear effect |
| Add timestep-aware MSE weighting to the BDM loss (ablation) | BDM · Latent spaces higher than 2×256 | FID | Helped |
| Add timestep-aware MSE weighting to the BDM loss (ablation) | BDM · Latent spaces higher than 2×256 | R-Precision Top-1 | Helped |
Surprising findings
- Simply increasing the latent dimension of a single diffusion model degrades R-Precision, motivating the split between basic and advanced models.
- The MModality metric for B2A-HDM on HumanML3D (1.914) is not best; the paper explicitly notes it only achieves comparable diversity and modality scores.
Limitations
- Performance degrades for text descriptions with unusual styles (very concrete or brief) given limited text-annotated training data.
- Focuses on the articulated body and neglects facial expressions and hand movements.
- Full model has 487.9M parameters, substantially larger than some baselines.
Experimental setup
Datasets: HumanML3D KIT-ML
Benchmarks: HumanML3D KIT-ML
Metrics: FID R-Precision MM-Dist Diversity MModality
Baselines: MDM MotionDiffuse MLD T2M-GPT TM2T T2M
Builds on: MLD classifier-free guidance denoising diffusion probabilistic models CLIP
Closest papers by meaning
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.90)
- SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing (CVPR 2025 · similarity 0.90)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.90)
- AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion (AAAI 2024 · similarity 0.90)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.89)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.