Research map / Video / Video generation
Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffKFC, a conditional diffusion model trained with sparse keyframes plus text for text-driven human motion synthesis, using dilated mask attention and a DCT-based transition smoothness prior.
Problem
Text-driven motion diffusion models give only coarse semantic control and cannot reliably reproduce animator-specified posture details; naive inference-time keyframe editing produces discontinuities and ignores sparse keyframes.
Approach
Train a diffusion transformer explicitly conditioned on sparse keyframes and text (rather than injecting keyframes only at inference), with a dilated mask attention keyframe encoder that gradually propagates valid keyframe tokens to the whole sequence, plus a DCT-based transition guidance at inference and classifier-free guidance over keyframe intensity.
Key results
- DiffKFC with 5% keyframes reaches FID 0.111 on HumanML3D versus 0.433 for PhysDiff, a reported 74.4% improvement.
- On KIT, DiffKFC achieves FID 0.164 versus MLD's 0.404, a reported 59.4% improvement.
- Keyframe rate ablation shows FID improves from 0.597 (0% keyframes) to 0.253 (2%), 0.111 (5%) and 0.079 (10%) on HumanML3D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffKFC instead of the strongest listed baseline, GMD | HumanML3D text-to-motion generation · HumanML3D | FID: 0.212 -> 0.111 | Helped |
| use DiffKFC instead of MLD | KIT text-to-motion generation · KIT | FID: 0.404 -> 0.164 | Helped |
| use DiffKFC instead of MLD | HumanML3D text-to-motion generation · HumanML3D | R-Precision (Top-3): 0.772 -> 0.686 | Hurt |
| use DiffKFC instead of MLD | KIT text-to-motion generation · KIT | R-Precision (Top-3): 0.734 -> 0.420 | Hurt |
| add transition guidance (ablation) | DiffKFC · HumanML3D, 5% keyframe rate | K-TranS: 0.161 -> 0.136 | Helped |
| replace DMA stack with a vanilla transformer encoder (ablation) | DiffKFC · HumanML3D | FID: 0.111 -> 0.477 | Hurt |
| encode keyframes and diffused target frames together in a unified encoder (ablation) | DiffKFC · HumanML3D | FID: 0.111 -> 0.293 | Hurt |
| increase keyframe rate from 5% to 10% (ablation) | DiffKFC · HumanML3D | FID: 0.111 -> 0.079 | Helped |
| reduce keyframe rate from 5% to 2% (ablation) | DiffKFC · HumanML3D | FID: 0.111 -> 0.253 | Hurt |
| remove keyframes (0% keyframe rate) (ablation) | DiffKFC · HumanML3D | FID: 0.111 -> 0.597 | Hurt |
| add keyframes at 5% rate rather than no keyframes (ablation) | DiffKFC · HumanML3D | ADE: 1.831 -> 0.205 | Helped |
| add keyframes at 5% rate rather than no keyframes (ablation) | DiffKFC · HumanML3D | K-Err: 1.829 -> 0.041 | Helped |
Surprising findings
- Naive inference-time keyframe editing (MDM-inp, MDM-grad) can hurt quality, with MDM-inp producing sudden pose jumps and MDM-grad failing to match keyframes.
- Replacing the keyframe encoder with a plain vanilla transformer or a unified encoder markedly degrades FID and keyframe error.
Limitations
- Inference is slower than state-of-the-art models due to keyframe encoding and about 1,000 reverse diffusion steps.
- Diversity and R-Precision on KIT and HumanML3D are not uniformly best; R-Precision on KIT is lower than MLD.
Experimental setup
Datasets: HumanML3D KIT Motion-Language
Benchmarks: HumanML3D KIT
Metrics: FID R-Precision Diversity ADE K-Err K-TranS
Baselines: JL2P Hier T2M MoFusion MLD PhysDiff GMD MDM MDM-inp MDM-grad
Builds on: MDM classifier-free guidance classifier guidance CLIP
Closest papers by meaning
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.90)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.90)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.89)
- HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models (CVPR 2024 · similarity 0.89)
- Dynamic Motion Blending for Versatile Motion Editing (CVPR 2025 · similarity 0.89)
- TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.