Research map / Video / Video generation
MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MotionMix, a weakly-supervised diffusion method that trains motion generators on a mix of noisy annotated and clean unannotated motions by splitting denoising steps at a pivot T*, and reports gains over fully supervised MDM, MotionDiffuse and EDGE.
Problem
Conditional 3D human motion generation normally requires large, clean, carefully annotated motion corpora, which are expensive to collect; the paper asks whether diffusion models can be trained from noisy annotated and unannotated motions instead.
Approach
Split the diffusion timesteps by data source: noisy annotated motions supervise the high-noise steps [T*+1, T] with conditions, while clean unannotated motions supervise [1, T*] with the condition dropped; at sampling, conditional guidance is used for the first T-T* steps and unconditional refinement for the last T*.
Key results
- On HumanML3D, MDM (MotionMix) reduces FID from 0.544 to 0.381 and raises R Precision from 0.611 to 0.632 relative to fully supervised MDM.
- On UESTC, MDM (MotionMix) improves FID from 12.81 to 11.40 versus MDM.
- On AIST++, EDGE (MotionMix) reports worse PFC (1.988 vs 1.605) but higher Beat Alignment (0.256 vs 0.224) than the re-trained EDGE baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MotionMix training with MDM instead of fully supervised MDM | MDM · HumanML3D | FID: 0.544^{±.440} -> 0.381^{±.042} | Helped |
| Use MotionMix training with MotionDiffuse instead of fully supervised MotionDiffuse | MotionDiffuse · HumanML3D | FID: 0.630^{±.001} -> 1.021^{±.071} | Hurt |
| Use MotionMix training with MDM instead of fully supervised MDM | MDM · KIT-ML | FID: 0.497^{±.021} -> 0.322^{±.020} | Helped |
| Use MotionMix training with MDM instead of fully supervised MDM | MDM · HumanAct12 | FID: 0.100^{±.000} -> 0.196^{±.007} | Hurt |
| Use MotionMix training with MDM instead of fully supervised MDM | MDM · UESTC | FID: 12.810^{±1.460} -> 11.400^{±0.393} | Helped |
| Use MotionMix training with EDGE instead of retrained EDGE | EDGE · AIST++ | Beat Alignment: 0.224^{±.025} -> 0.256^{±.013} | Helped |
| Set pivot to T*=0 rather than T*=60 (ablation) | MotionMix · HumanML3D | FID: 0.381^{±.042} -> 0.714^{±.045} | Hurt |
| Set pivot to T*=20 rather than T*=60 (ablation) | MotionMix · HumanML3D | FID: 0.381^{±.042} -> 0.497^{±.048} | Hurt |
| Set pivot to T*=40 rather than T*=60 (ablation) | MotionMix · HumanML3D | FID: 0.381^{±.042} -> 0.402^{±.032} | Hurt |
| Set pivot to T*=80 rather than T*=60 (ablation) | MotionMix · HumanML3D | FID: 0.381^{±.042} -> 0.589^{±.059} | Hurt |
| Increase noisy-data ratio from 30% to 50% (ablation) | MotionMix · HumanML3D | FID: 0.898^{±.045} -> 0.381^{±.042} | Helped |
| Increase noisy-data ratio from 50% to 70% (ablation) | MotionMix · HumanML3D | FID: 0.381^{±.042} -> 0.359^{±.030} | Helped |
Surprising findings
- MotionMix trained on half-noisy data often matches or beats fully supervised backbones on FID and R Precision, which the authors attribute to better generalizability.
- On HumanAct12, MDM (MotionMix) substantially degrades FID (0.196 vs 0.100) and accuracy, which the authors attribute to the small dataset size.
Limitations
- Noise is synthetically approximated with Gaussian noise rather than real mocap corruption, and the noise schedule range is chosen empirically.
- Performance is inconsistent across tasks; MotionDiffuse (MotionMix) is worse than its original on HumanML3D FID and HumanAct12 degrades.
- The pivot T* and noisy ranges are tuned by hyperparameter search per dataset.
Experimental setup
Datasets: HumanML3D KIT-ML HumanAct12 UESTC AIST++ AMASS
Benchmarks: HumanML3D KIT-ML HumanAct12 UESTC AIST++
Metrics: R Precision (top 3) FID Multimodal Distance Diversity MultiModality Accuracy PFC Beat Alignment Dist_k Dist_g
Baselines: Language2Pose Text2Gestures MLD Action2Motion ACTOR INR Bailando FACT
Builds on: DDPM classifier-free guidance MDM MotionDiffuse EDGE
Closest papers by meaning
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.91)
- MixerMDM: Learnable Composition of Human Motion Diffusion Models (CVPR 2025 · similarity 0.90)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.90)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.89)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.89)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.