Research map / Probabilistic and scientific modeling / Dynamical systems
Homogeneous Dynamics Space for Heterogeneous Humans
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HDyS, a homogeneous latent space for human dynamics that aggregates heterogeneous kinematic and dynamic representations and datasets (biomechanics and RL-simulated) via inverse-forward dynamics auto-encoders trained with reconstruction and InfoNCE alignment losses.
Problem
Human dynamics (joint torques, muscle actions, sEMG) is underexplored because existing data and representations are heterogeneous in both kinematics/dynamics formats and in domains (biomechanics vs. RL-simulated), hindering unified data-driven learning.
Approach
Build a shared latent space by aggregating heterogeneous kinematics (markers, keypoints, joint angles, SMPL) and dynamics (joint torques, muscle actions, sEMG) through inverse- and forward-dynamics auto-encoders, aligned with a contrastive InfoNCE loss so that representation and dataset heterogeneity are unified into homogeneous knowledge.
Key results
- HDyS achieves ImDy mPJE 0.4674 (best representation) vs 0.6300 for ImDyS; AddBiomechanics mPJE 0.1189 avg / 0.1243 best, improving over prior methods.
- On GroundLink, finetuned HDyS reduces left-foot mPJE to 0.0514 from 0.0711 for GroundLinkNet.
- In PHC character control, adding HDyS latents lowers global mPJPE from 74.514 to 72.692.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine heterogeneous dynamics datasets in HDyS rather than train on one dataset (ablation) | HDyS · ImDy | mPJE: 0.6854 -> 0.5765 | Helped |
| Combine heterogeneous dynamics datasets in HDyS rather than train on one dataset (ablation) | HDyS · AddBiomechanics | mPJE: 0.1695 -> 0.1189 | Helped |
| Remove ImDy data (ablation) | HDyS · MiA | RMSE: 11.8/11.6 -> 15.8/15.6 | Hurt |
| Remove AddBiomechanics data (ablation) | HDyS · MiA | RMSE: 11.8/11.6 -> 17.1/17.0 | Hurt |
| Remove MinT data (ablation) | HDyS · MiA | RMSE: 11.8/11.6 -> 16.9/16.8 | Hurt |
| Remove AMASS kinematics data (ablation) | HDyS · MiA | RMSE: 11.8/11.6 -> 14.7/14.9 | Hurt |
| Remove cross-representation alignment loss (ablation) | HDyS · ImDy | mPJE: 0.5765/0.4674 -> 0.6575/0.5019 | Hurt |
| Remove forward-dynamics auto-encoder (ablation) | HDyS · AddBiomechanics | mPJE: 0.1189/0.1243 -> 0.1198/0.1261 | Hurt |
| Reduce latent dimension to 32 (ablation) | HDyS · MinT | RMSE: 0.0614/0.0615 -> 0.0650/0.0648 | Hurt |
| Add HDyS keypoint latents as observations | physical character control baseline · Physical character control | mPJPEg: 74.514 -> 72.692 | Helped |
| Fine-tune HDyS on GroundLink subjects 1-6 | HDyS · GroundLink subject 7 | L-Foot mPJE: 0.0584 -> 0.0514 | Helped |
Surprising findings
- Averaging predictions across kinematic representations sometimes hurts (e.g., ImDy and MiA averages worse than best representation), while MinT and AddBiomechanics benefit from averaging.
- Biomechanical simulation with HDyS-predicted torques gave lower per-frame MSE than optimized torques at k=1 to 5 steps (e.g., 0.00061 vs 0.00063 at k=1), which the authors describe as surprising.
Limitations
- Imbalanced focus on lower-body dynamics in the datasets leads to worse upper-body predictions.
- Muscle action magnitudes can diverge from ground truth and minor changes are omitted.
- Forward-dynamics compatibility limits accuracy in long-horizon biomechanical simulation, with drift accumulating as k increases.
Experimental setup
Datasets: AddBiomechanics MinT MiA ImDy AMASS GroundLink
Benchmarks: ImDy AddBiomechanics MinT MiA GroundLink PHC character control
Metrics: mPJE RMSE PCC MSE mPJPE Evel Eacc
Baselines: ImDyS MiA GroundLinkNet PHC baseline Optimized torques (Nimble)
Builds on: InfoNCE PHC AddBiomechanics MinT
Closest papers by meaning
- Bringing Motion Taxonomies to Continuous Domains via GPLVM on Hyperbolic manifolds (ICML 2024 · similarity 0.88)
- ImDy: Human Inverse Dynamics from Imitated Observations (ICLR 2025 · similarity 0.87)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.87)
- SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model (CVPR 2024 · similarity 0.86)
- TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation (CVPR 2025 · similarity 0.86)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.