Research map / Video / Video generation
MixerMDM: Learnable Composition of Human Motion Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MixerMDM, a learnable Transformer-based Mixer trained adversarially (one discriminator per pre-trained model) to dynamically combine denoising outputs of pre-trained individual and interaction human motion diffusion models.
Problem
Combining pre-trained text-conditioned human motion diffusion models for fine-grained control of individual motions within human-human interactions, where prior mixing strategies use manually set, static weights.
Approach
A lightweight Mixer (21M params) predicts per-step, per-motion mixing weights (global, temporal, spatial, or spatio-temporal) from both models' outputs, conditions, and timestep, trained with hinge-loss adversarial discriminators per pre-trained model and no ground-truth mixed motions.
Key results
- ST-type Mixer with in2IN interaction and individual models achieves the highest Overall Alignment of .335±.01, versus .221 for DiffusionBlending and .217 for DualMDM.
- In a user study with 35 participants, MixerMDM ranked first 85.14% of the time for interaction alignment versus 10.29% for DualMDM.
- Swapping in a weaker MDM individual model with Mixer weights trained on the top models improves Overall Alignment by 35% (ST) without additional training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace manually specified fixed mixing weights with learned spatio-temporal weights | MixerMDM using in2IN interaction and individual models · Overall Alignment | Overall Alignment: .217±.00 -> .335±.01 | Helped |
| Use global rather than spatio-temporal mixing weights (ablation) | MixerMDM using in2IN interaction and individual models · Overall Alignment | Overall Alignment: .335±.01 -> .317±.00 | Hurt |
| Use temporal rather than spatio-temporal mixing weights (ablation) | MixerMDM using in2IN interaction and individual models · Overall Alignment | Overall Alignment: .335±.01 -> .245±.01 | Hurt |
| Use spatial rather than spatio-temporal mixing weights (ablation) | MixerMDM using in2IN interaction and individual models · Overall Alignment | Overall Alignment: .335±.01 -> .310±.00 | Hurt |
| Use MDM instead of in2IN as the individual-motion model (ablation) | MixerMDM with global mixing weights · Overall Alignment | Overall Alignment: .317±.00 -> .250±.01 | Hurt |
| Use MDM instead of in2IN as the individual-motion model (ablation) | MixerMDM with temporal mixing weights · Overall Alignment | Overall Alignment: .245±.01 -> .267±.00 | Helped |
| Use MDM instead of in2IN as the individual-motion model (ablation) | MixerMDM with spatial mixing weights · Overall Alignment | Overall Alignment: .310±.00 -> .244±.01 | Hurt |
| Use MDM instead of in2IN as the individual-motion model (ablation) | MixerMDM with spatio-temporal mixing weights · Overall Alignment | Overall Alignment: .335±.01 -> .288±.00 | Hurt |
| Use InterGen instead of in2IN as the interaction model (ablation) | MixerMDM with global mixing weights · Overall Alignment | Overall Alignment: .317±.00 -> .317±.00 | No clear effect |
| Use InterGen instead of in2IN as the interaction model (ablation) | MixerMDM with temporal mixing weights · Overall Alignment | Overall Alignment: .245±.01 -> .312±.01 | Helped |
| Use InterGen instead of in2IN as the interaction model (ablation) | MixerMDM with spatial mixing weights · Overall Alignment | Overall Alignment: .310±.00 -> .273±.01 | Hurt |
| Use InterGen instead of in2IN as the interaction model (ablation) | MixerMDM with spatio-temporal mixing weights · Overall Alignment | Overall Alignment: .335±.01 -> .293±.00 | Hurt |
Surprising findings
- The most adaptable Mixer variants had much lower alignment, suggesting a trade-off between adaptability and mixing quality.
Limitations
- Adds inference-time computation because mixing weights are computed at every denoising step.
- Adversarial training is unstable and hyperparameters are hard to tune.
- Requires models to share a unified data representation, sometimes necessitating retraining.
Experimental setup
Datasets: InterHuman HumanML3D
Benchmarks: InterHuman HumanML3D
Metrics: R-Precision (Top-3) alignment Overall Alignment (harmonic mean) Adaptability (std of mixing weights) user study average rank and first-rank rate
Baselines: DiffusionBlending DualMDM Finetuned in2IN
Builds on: DualMDM DiffusionBlending GAN MDM in2IN
Closest papers by meaning
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.92)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.90)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.88)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.88)
- Seamless Human Motion Composition with Blended Positional Encodings (CVPR 2024 · similarity 0.88)
- Optimizing Diffusion Noise Can Serve As Universal Motion Priors (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.