Research map / Video / Video generation
Seamless Human Motion Composition with Blended Positional Encodings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FlowMDM, a bidirectional diffusion transformer for seamless long human motion composition from varying text, using Blended Positional Encodings (absolute-then-relative) and Pose-Centric Cross-Attention, plus two jerk-based transition metrics (PJ, AUJ).
Problem
Generating long continuous human motion sequences driven by a series of varying textual descriptions with seamless transitions, without redundant denoising or postprocessing, and with metrics sensitive to abrupt transitions.
Approach
Blend absolute positional encoding early in the denoising chain (for global coherence) with rotary relative positional encoding later (for translation-invariant smooth transitions), and inject each frame's condition into queries only (PCCAT) so a model trained with one condition per sequence generalizes to multiple conditions at inference.
Key results
- On HumanML3D, FlowMDM reaches subsequence R-prec 0.685 and transition FID 1.38 (vs DoubleTake 4.19), with AUJ 0.51 vs 0.64 for the best prior DoubleTake.
- On Babel, FlowMDM achieves subsequence FID 0.99 and transition AUJ 0.13 versus 0.64 for DoubleTake, matching state-of-the-art accuracy.
- FlowMDM uses 47.1%, 28.4%, and 16.5% fewer pose-wise denoising steps than DoubleTake, DiffCollage, and MultiDiffusion, respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use blended positional encodings instead of absolute-only encodings (ablation) | FlowMDM on Babel · Babel subsequences | R-prec: 0.699±0.004 -> 0.702±0.004 | Helped |
| Use blended positional encodings instead of relative-only encodings (ablation) | FlowMDM on Babel · Babel subsequences | R-prec: 0.635±0.006 -> 0.702±0.004 | Helped |
| Use blended positional encodings instead of absolute-only encodings (ablation) | FlowMDM on HumanML3D · HumanML3D subsequences | R-prec: 0.689±0.005 -> 0.685±0.004 | No clear effect |
| Use blended positional encodings instead of relative-only encodings (ablation) | FlowMDM on HumanML3D · HumanML3D subsequences | R-prec: 0.531±0.005 -> 0.685±0.004 | Helped |
| Use blended positional encodings at inference instead of absolute encodings after blended-encoding training (ablation) | FlowMDM on Babel · Babel subsequences | R-prec: 0.716±0.006 -> 0.702±0.004 | Hurt |
| Use relative positional encodings at inference instead of absolute encodings after blended-encoding training (ablation) | FlowMDM on Babel · Babel transitions | Peak Jerk: 1.78±0.01 -> 0.06±0.00 | Helped |
| Use blended positional encodings at inference instead of absolute encodings after blended-encoding training (ablation) | FlowMDM on HumanML3D · HumanML3D subsequences | R-prec: 0.699±0.005 -> 0.685±0.004 | Hurt |
| Use relative positional encodings at inference instead of absolute encodings after blended-encoding training (ablation) | FlowMDM on HumanML3D · HumanML3D transitions | FID: 2.42±0.09 -> 1.38±0.05 | Helped |
| Replace PCCAT with self-attention (ablation) | FlowMDM on Babel · Babel transitions | FID: 2.61±0.06 -> 1.91±0.03 | Hurt |
| Replace PCCAT with cross-attention (ablation) | FlowMDM on Babel · Babel transitions | FID: 2.61±0.06 -> 2.57±0.08 | No clear effect |
| Replace PCCAT with self-attention (ablation) | FlowMDM on HumanML3D · HumanML3D transitions | FID: 1.38±0.05 -> 3.19±0.08 | Hurt |
| Replace PCCAT with cross-attention (ablation) | FlowMDM on HumanML3D · HumanML3D transitions | FID: 1.38±0.05 -> 3.93±0.14 | Hurt |
Surprising findings
- Training with blended PE also improves the APE-only and RPE-only sampling results, not just the blended sampling.
- The FID score is not correlated with the proposed AUJ metric, suggesting FID alone misses transition abruptness.
Limitations
- Absolute stage does not model relationships between subsequences, so low-frequency structure is generated independently.
- Jerk peak remains in FlowMDM composition transitions on Babel relative to ground truth.
- Evaluated only on single-human motion from text on Babel and HumanML3D.
Experimental setup
Datasets: Babel HumanML3D
Benchmarks: Babel HumanML3D
Metrics: R-precision MM-Dist FID diversity Peak Jerk Area Under the Jerk
Baselines: TEACH TEACH B DoubleTake DoubleTake* DiffCollage MultiDiffusion
Builds on: MDM RoPE DoubleTake classifier-free guidance sinusoidal positional encoding
Closest papers by meaning
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.90)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.89)
- MMM: Generative Masked Motion Model (CVPR 2024 · similarity 0.89)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.89)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.89)
- Towards Detailed Text-to-Motion Synthesis via Basic-to-Advanced Hierarchical Diffusion Model (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.