Research map / Video / Video generation
Human Motion Diffusion as a Generative Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that a pretrained text-to-motion diffusion model (MDM) can be reused as a frozen prior via three composition methods (DoubleTake for long sequences, ComMDM for two-person motion, DiffusionBlending for joint/trajectory control) that need little or no new data.
Problem
Human motion diffusion models are limited by scarce annotated motion data, focus on short single-person clips, and offer coarse control; the paper asks how a pretrained motion prior can enable long sequences, two-person interaction, and fine-grained control with little data.
Approach
Compose frozen or lightly fine-tuned diffusion priors: DoubleTake generates overlapping intervals in one batch with handshake averaging and soft-masked second-take refinement (zero-shot); ComMDM adds a small transformer communication block between two frozen MDMs (few-shot); DiffusionBlending interpolates the noise predictions of models fine-tuned for different joint controls, generalizing classif
Key results
- On BABEL, DoubleTake transition FID 1.88 (70-frame margin) vs TEACH 3.86, and motion FID 1.04 vs TEACH 1.12.
- On HumanML3D trajectory control, fine-tuned MDM reaches FID 0.54 vs 0.98 for inpainted MDM; DiffusionBlending for left wrist+trajectory reaches FID 0.22 vs 1.18.
- In a 30-participant user study, ComMDM was preferred over MRT and MDM for two-person prefix completion, though MRT had lower L2 error (0.146 vs 0.176 joints at 3s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the second take to refine transitions (ablation) | DoubleTake · HumanML3D test | FID: 1.00 -> 0.60 | Helped |
| Use the first take before the second take (ablation) | DoubleTake · HumanML3D test | FID: 1.09 -> 0.60 | Helped |
| Set transition mask ramp length b to 20 instead of the default (ablation) | DoubleTake · HumanML3D test | FID: 0.60 -> 0.84 | Hurt |
| Set handshake size h to 30 instead of the default (ablation) | DoubleTake · HumanML3D test | FID: 0.60 -> 1.03 | Hurt |
| Set handshake size h to 40 instead of the default (ablation) | DoubleTake · HumanML3D test | FID: 0.60 -> 1.16 | Hurt |
| Set soft mask M_soft to 0.0 instead of the default (ablation) | DoubleTake · HumanML3D test | FID: 0.60 -> 0.85 | Hurt |
| Set soft mask M_soft to 0.2 instead of the default (ablation) | DoubleTake · HumanML3D test | FID: 0.60 -> 0.90 | Hurt |
| Add the ComMDM communication block to the fixed MDM priors (ablation) | ComMDM · 3DPW prefix completion, 3s | Root error [m]: 0.54 -> 0.30 | Helped |
| Use 2 or 4 ComMDM layers instead of 1 (ablation) | ComMDM · 3DPW prefix completion, 3s | Root error [m]: 0.30 -> 0.31; 0.30 -> 0.32 | Hurt |
| Place the communication block at a lower transformer layer instead of layer 8 (ablation) | ComMDM · 3DPW prefix completion, 3s | Root error [m]: 0.30 -> 0.33; 0.30 -> 0.36; 0.30 -> 0.45; 0.30 -> 0.47 | Hurt |
| Fine-tune MDM for trajectory control rather than using original inpainting | MDM · HumanML3D test, trajectory control | FID: 0.98 -> 0.54 | Helped |
| Fine-tune MDM for left wrist control rather than using original inpainting | MDM · HumanML3D test, left wrist control | FID: 0.82 -> 0.34 | Helped |
Surprising findings
- MRT achieves lower L2 prefix-completion error than ComMDM, yet produces static, unrealistic motion, so the authors argue L2 metrics are misleading here.
- DoubleTake produces coherent long motions (~10 minutes) from a prior trained only on clips up to ~10 seconds, with transitions never annotated in training data.
Limitations
- ComMDM generalizes only to interaction types seen during training and was trained on very few examples.
- Long-sequence quality is bounded by the base MDM and motion may be inconsistent between distant intervals.
- Two-person generation does not model valid contacts between people.
Experimental setup
Datasets: HumanML3D BABEL 3DPW AMASS
Benchmarks: HumanML3D BABEL 3DPW
Metrics: R-precision FID diversity multimodal distance L2 joint and root error user study preference
Baselines: TEACH MRT MDM inpainting MDM (no Com) Com only
Builds on: Motion Diffusion Model (MDM) classifier-free guidance DDPM MRT
Closest papers by meaning
- MixerMDM: Learnable Composition of Human Motion Diffusion Models (CVPR 2025 · similarity 0.92)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.91)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.90)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.90)
- Optimizing Diffusion Noise Can Serve As Universal Motion Priors (CVPR 2024 · similarity 0.89)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.