Research map / Video / Video generation
Single Motion Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SinMDM, a lightweight shallow UNet with QnA local attention diffusion model that learns from a single motion sequence and supports inference-time motion composition, harmonization/style transfer, long-motion and crowd synthesis.
Problem
Generating diverse, faithful motion for characters with arbitrary skeletons (animals, creatures) where only a single motion sequence is available for training.
Approach
Use a DDPM that predicts clean motion with a shallow UNet whose global attention is replaced by QnA local shift-invariant attention, giving a narrow receptive field that avoids overfitting on a single sequence and enables many applications at inference time without retraining.
Key results
- On the Mixamo benchmark SinMDM reaches a harmonic mean of 0.85 vs -0.22 for Ganimator, with 5.26M parameters vs 21.7M and 1.5h total time vs 6.0h.
- On the Gangnam-style motion SinMDM achieves the best coverage (98.1) and harmonic mean (0.39) among compared methods, with Ganimator at 97.2 and 0.38.
- A user study with 10 users per comparison over 8 motions significantly preferred SinMDM over Ganimator and MDM on crops.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SinMDM instead of Ganimator | Single-motion synthesis · Mixamo benchmark | Harmonic mean: -0.22 -> 0.85 | Helped |
| Use SinMDM instead of Ganimator | Single-motion synthesis · Gangnam-style motion | Harmonic mean: 0.38 -> 0.39 | Helped |
| Use SinMDM instead of MDM trained on a single motion | Single-motion synthesis · HumanML3D | Harmonic mean: 0.14 -> 0.82 | Helped |
| Use SinMDM instead of MDM trained on short crops | Single-motion synthesis · HumanML3D | Harmonic mean: -1.01 -> 0.82 | Helped |
| Add a narrow receptive field (depth 1) rather than a wide one (depth 3) (ablation) | QnA-based UNet · HumanML3D | Inter diversity: 0.20 -> 0.73 | Helped |
| Add QnA local attention instead of vanilla attention (ablation) | UNet, depth 1 · HumanML3D | Harmonic mean: 0.79 -> 0.82 | Helped |
| Add QnA attention instead of using no attention (ablation) | UNet, depth 1 · HumanML3D | Harmonic mean: 0.79 -> 0.82 | Helped |
| Replace the QnA-based UNet with a QnA-based transformer (ablation) | SinMDM · HumanML3D | SiFID: 1.87 -> 5.99 | Hurt |
| Replace the QnA-based UNet with a QnA-based transformer (ablation) | SinMDM · HumanML3D | Inter diversity: 0.73 -> 1.74 | Helped |
Surprising findings
- Vanilla MDM trained on a single motion overfits completely (SiFID 0.01), while MDM on short crops diverges from the input motion (SiFID 13.94).
Limitations
- Inherent inability to synthesize out-of-distribution motion when learning from a single instance.
- Relatively long inference time due to iterative diffusion sampling.
Experimental setup
Datasets: HumanML3D Mixamo Truebones Zoo Gangnam-style motion
Benchmarks: Mixamo benchmark HumanML3D benchmark Gangnam-style motion
Metrics: coverage global diversity local diversity inter diversity intra diversity diff SiFID harmonic mean parameters iteration time total time
Baselines: Ganimator MDM MDM on crops MotionTexture acRNN
Builds on: DDPM QnA MDM Ganimator UNet
Closest papers by meaning
- AAMDM: Accelerated Auto-regressive Motion Diffusion Model (CVPR 2024 · similarity 0.88)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.88)
- How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects (ICML 2025 · similarity 0.88)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.87)
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning (ICLR 2024 · similarity 0.87)
- AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.