Research map / 3D vision / 3D generation
Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SkeletonDiffusion, a GCN-based latent diffusion model for stochastic 3D human motion prediction using a nonisotropic Gaussian diffusion whose noise covariance follows the skeleton graph.
Problem
Probabilistic human motion prediction produces diverse futures but often with limb stretching and jitter (bone-length inconsistency), which current diversity-favoring models fail to penalize.
Approach
Replaces isotropic i.i.d. diffusion noise with a fixed non-diagonal covariance derived from the skeleton adjacency matrix (with a scheduler moving from isotropic to nonisotropic), applied in a latent space whose joint dimensions correspond to body joints, with a fully GCN encoder/denoiser and a min-reconstruction-loss sample selection.
Key results
- On AMASS, SkeletonDiffusion achieves ADE 6.124 and FDE 0.561 versus CoMusion's 6.715 and 0.469, with the lowest limb stretching (str 3.15 mean) and jitter (0.20) among compared methods.
- On FreeMan, it reaches the best precision (mean 0.374, RMSE 0.457) and diversity (APD 7.58) while keeping lower limb stretching than baselines.
- Zero-shot on 3DPW, it reports the best body realism (str 4.16, jit 0.23) and best ADE (6.025) and mean error (0.472).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SkeletonDiffusion instead of CoMusion | stochastic human motion prediction · AMASS | ADE: 6.715 -> 6.124 | Helped |
| Use SkeletonDiffusion instead of CoMusion | stochastic human motion prediction · AMASS | str: 0.25 -> 0.20 | Helped |
| Use SkeletonDiffusion instead of CoMusion | stochastic human motion prediction · AMASS | APD: 4.04 -> 3.15 | Hurt |
| Use SkeletonDiffusion instead of CoMusion | stochastic human motion prediction · FreeMan | ADE: 7.812 -> 7.424 | Helped |
| Use SkeletonDiffusion instead of BeLFusion | stochastic human motion prediction · FreeMan | str: 0.41 -> 0.51 | Hurt |
| Use SkeletonDiffusion instead of BeLFusion | stochastic human motion prediction · 3DPW zero-shot | str: 0.22 -> 0.17 | Helped |
| Use SkeletonDiffusion instead of DivSampling | stochastic human motion prediction · 3DPW zero-shot | APD: 11.04 -> 3.02 | Hurt |
| Add Typed-Graph Attention layers (ablation) | SkeletonDiffusion GCN architecture · AMASS | performance metrics | Helped |
| Add the nonisotropic covariance scheduler γt (ablation) | SkeletonDiffusion diffusion process · AMASS | performance metrics | Helped |
| Replace isotropic diffusion with the simple nonisotropic variant of Eq. (3) (ablation) | SkeletonDiffusion diffusion process · AMASS | performance metrics | Helped |
Surprising findings
- Diversity metric APD can be inflated by limb stretching and jitter artifacts, so high-diversity VAE methods show the worst body realism.
- Diverse predictions from DivSampling are often semantically inconsistent (e.g., cartwheeling) despite high APD.
Limitations
- Experiments restricted to standard human skeletons without fine-grained joints such as fingers or face.
- Evaluating stochastic motion prediction, especially diversity, remains an open challenge.
- Nonisotropic formulation applied only to a fixed skeleton graph; covariance not learned from input.
Experimental setup
Datasets: AMASS FreeMan 3DPW
Benchmarks: AMASS FreeMan 3DPW
Metrics: ADE FDE MMADE MMFDE APD APDE CMD MAE limb stretching (str) limb jitter (jit)
Baselines: Zero-Velocity TPK DLow GSPS DivSamp HumanMAC BeLFusion CoMusion
Builds on: DDPM latent diffusion Typed-Graph Convolutions BeLFusion CoMusion
Closest papers by meaning
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects (NeurIPS 2025 · similarity 0.87)
- Human Motion Diffusion as a Generative Prior (ICLR 2024 · similarity 0.86)
- Existence Is Chaos: Enhancing 3D Human Motion Prediction with Uncertainty Consideration (AAAI 2024 · similarity 0.86)
- Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoiser (AAAI 2024 · similarity 0.86)
- Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction (NeurIPS 2024 · similarity 0.86)
- DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.