Research map / Video / Video generation
Deterministic-to-Stochastic Diverse Latent Feature Mapping for Human Motion Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DSDFM, a two-stage VQVAE plus flow-style deterministic ODE (DerODE) and stochastic SDE sampler (DivSDE) for diverse, straight-trajectory human motion synthesis on HumanAct12 and HumanML3D.
Problem
Score-based generative models for human motion synthesis have curved diffusion trajectories, leading to unstable training, slow sampling, and a trade-off between diversity and fidelity.
Approach
Learn a VQVAE latent space of motions, then map a standard Gaussian to it with an OT-coupled straight-path ODE (DerODE) trained with drift-estimate and drift-consistency losses, and add a noise-controlled SDE sampler (DivSDE) that reuses the ODE output as the score estimate, so diversity is gained without extra training.
Key results
- Unconditional generation on HumanAct12: FID 12.86 and Diversity 18.41, versus 13.03 and 17.57 for the next best (Modi), with 15M parameters.
- Action-to-Motion on HumanAct12: FID 0.068 versus 0.070 for MotionDiffuse, with diversity 6.851 slightly below MotionDiffuse's 6.850 at the 0.01% level reported as a near tie.
- Training time on HumanAct12 at 500 epochs is 25.33 minutes versus 42.93 (VPSDE) and 40.57 (VESDE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DSDFM instead of Modi | unconditional human motion synthesis · HumanAct12 | FID: 13.03 -> 12.86 | Helped |
| use DSDFM instead of Modi | unconditional human motion synthesis · HumanAct12 | Diversity: 17.57 -> 18.41 | Helped |
| use DSDFM instead of MotionDiffuse | action-conditioned motion generation · HumanAct12 | FID: 0.070±.000 -> 0.068±.010 | Helped |
| use DSDFM instead of MotionDiffuse | action-conditioned motion generation · HumanAct12 | Diversity: 6.850±.020 -> 6.851±.008 | No clear effect |
| replace DSDFM with VPSDE (ablation) | human motion generation · HumanAct12, 1000 steps | FID: 12.24 -> 14.31 | Hurt |
| replace DSDFM with VESDE (ablation) | human motion generation · HumanAct12, 1000 steps | FID: 12.24 -> 14.17 | Hurt |
| replace DSDFM with VPSDE (ablation) | human motion generation · HumanML3D | FID: 0.054±.010 -> 0.080±.024 | Hurt |
| replace DSDFM with VESDE (ablation) | human motion generation · HumanML3D | FID: 0.054±.010 -> 0.078±.013 | Hurt |
| replace DivSDE with NCSN++ SDE (ablation) | DSDFM · HumanAct12, unconditional | Diversity: 18.41 -> 17.54 | Hurt |
| replace DivSDE with DDPM++ SDE (ablation) | DSDFM · HumanAct12, unconditional | FID: 12.86 -> 13.25 | Hurt |
| replace DivSDE with NCSN++ SDE (ablation) | DSDFM · HumanAct12, unconditional | FID: 12.86 -> 13.01 | Hurt |
Surprising findings
- The reported diversity of Action-to-Motion is marginally lower than MotionDiffuse (6.851 vs 6.850 is reported as a slight deficit), so the method is not uniformly best on all metrics.
Limitations
- Diversity gains over the best baselines are small, and the Action-to-Motion diversity is slightly below MotionDiffuse.
- Evaluated only on HumanAct12 and HumanML3D; no broader datasets or open-vocabulary settings.
- Hyperparameters such as diversity strength eta and interpolation loss weight were set by the authors with limited sensitivity analysis.
Experimental setup
Datasets: HumanAct12 HumanML3D
Benchmarks: HumanAct12 HumanML3D
Metrics: FID KID Precision Recall Accuracy Diversity Multimodality
Baselines: VPoser Action2Motion ACTOR MDM MLD Modi INR MotionDiffuse VPSDE VESDE
Builds on: VQVAE score-based generative models flow matching optimal transport DDIM
Closest papers by meaning
- Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction (NeurIPS 2024 · similarity 0.86)
- Seamless Human Motion Composition with Blended Positional Encodings (CVPR 2024 · similarity 0.86)
- HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback (AAAI 2024 · similarity 0.86)
- Motion Diversification Networks (CVPR 2024 · similarity 0.86)
- Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression (CVPR 2025 · similarity 0.86)
- Move-in-2D: 2D-Conditioned Human Motion Generation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.