Research map / 3D vision / Human pose estimation
MultiPhys: Multi-Person Physics-aware 3D Motion Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Corrects kinematic multi-person 3D motion estimates from monocular video by running them through a MuJoCo physics simulator with a PPO imitation policy and an iterative loop-N refinement, reducing penetration and ground penetration.
Problem
Recovering multi-person 3D human motion from monocular video in world coordinates is physically implausible for kinematic methods, which produce inter-person penetration, ground penetration, and foot skating, especially under close interaction and occlusion.
Approach
Feed kinematic SLAHMR estimates into a physics simulator (MuJoCo) with all persons simulated together in one scene, using a single-person imitation policy (re-purposed UHC) trained on AMASS, plus an autoregressive "loop-N" iterative refinement that lets simulated humanoids catch up with reference poses.
Key results
- On Hi4D, inter-person penetration drops from 367.3 (SLAHMR) to 51.1, and ground penetration from 12.2 to 2.4.
- On ExPI, inter-person penetration drops from 567.3 to 73.0 and ground penetration from 18.6 to 0.6, while W-MPJPE improves from 263.3 to 250.9.
- On CHI3D, W-MPJPE improves from 177.1 (SLAHMR) to 174.7 and penetration from 139.3 to 18.7, but skating is slightly worse (2.7 vs 1.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply physics-aware correction to SLAHMR motion | MultiPhys · CHI3D | Pen.: 139.3 -> 18.7 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · CHI3D | W-MPJPE: 177.1 -> 174.7 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · CHI3D | Skating: 1.0 -> 2.7 | Hurt |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · Hi4D | Pen.: 367.3 -> 51.1 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · Hi4D | Gnd Pen.: 12.2 -> 2.4 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · Hi4D | Acc. Error: 6.9 -> 9.6 | Hurt |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · ExPI | Pen.: 567.3 -> 73.0 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · ExPI | Gnd Pen.: 18.6 -> 0.6 | Helped |
| Apply physics-aware correction to SLAHMR motion | MultiPhys · ExPI | Acc. Error: 8.2 -> 17.1 | Hurt |
| Add loop-N iterative refinement (Nl=2) instead of Loop1 (ablation) | MultiPhys · Hi4D | W-MPJPE | Helped |
| Increase loop-N iterations beyond Nl=2 (ablation) | MultiPhys · Hi4D | W-MPJPE | Hurt |
| Increase loop-N iterations (ablation) | MultiPhys · Hi4D | Skating | Hurt |
Surprising findings
- Physics-based correction improves pose accuracy on CHI3D, not just physical plausibility metrics.
- Loop-N refinement increases foot skating as the number of iterations grows, a trade-off with physics metrics.
Limitations
- Performance depends on 2D keypoint detector quality since SLAHMR initialization fails when detection fails.
- Loop-N refinement slightly introduces foot skating and the best Nl varies by motion.
- Dynamic motions in Hi4D and ExPI are harder for the simulated agent to track, leaving pose accuracy close to SLAHMR.
Experimental setup
Datasets: AMASS CHI3D Hi4D ExPI
Benchmarks: CHI3D Hi4D ExPI
Metrics: W-MPJPE PA-MPJPE acceleration error foot skating ground penetration inter-person penetration
Baselines: SLAHMR EmbPose-MP
Builds on: SLAHMR Universal Humanoid Controller EmbPose SimPoE PPO
Closest papers by meaning
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.88)
- CoMotion: Concurrent Multi-person 3D Motion (ICLR 2025 · similarity 0.86)
- FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance (CVPR 2025 · similarity 0.85)
- PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos (CVPR 2024 · similarity 0.85)
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.85)
- PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.