Research map / 3D vision / Human pose estimation
WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
WHAM reconstructs world-grounded 3D human motion from monocular video online by fusing AMASS-pretrained 2D-keypoint motion features with image features, and recovers global trajectory using camera angular velocity and contact-aware refinement.
Problem
Estimating accurate, temporally smooth 3D human pose and shape in global world coordinates from monocular video captured by a moving camera, without foot sliding and without slow optimization pipelines.
Approach
Pretrain a uni-directional RNN motion encoder/decoder to lift synthetic 2D keypoint sequences (from AMASS with dynamic virtual cameras) to 3D, then jointly fine-tune with a feature integrator fusing image features; a trajectory decoder conditioned on camera angular velocity and a foot-contact-aware refiner produce global motion.
Key results
- On 3DPW, WHAM (ViT) reaches 57.8 MPJPE and 35.9 PA-MPJPE, outperforming prior per-frame and video-based methods on per-frame metrics.
- On EMDB 2 global trajectory, WHAM (w/ DROID) achieves WA-MPJPE100 of 133.3 mm versus 326.9 mm for SLAHMR and RTE of 4.6% versus 10.2%.
- WHAM runs at about 200 fps for the network excluding preprocessing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use WHAM (ViT) instead of ReFit | 3D human motion recovery · 3DPW | MPJPE: 65.3 -> 57.8 | Helped |
| use WHAM (ViT) instead of ReFit | 3D human motion recovery · RICH | MPJPE: 80.7 -> 80.0 | Helped |
| use WHAM (ViT) instead of ReFit | 3D human motion recovery · EMDB | MPJPE: 88.0 -> 79.7 | Helped |
| use WHAM with DPVO instead of SLAHMR | global trajectory recovery · EMDB 2 | WA-MPJPE100: 326.9 -> 135.6 | Helped |
| use WHAM with DPVO instead of SLAHMR | global trajectory recovery · EMDB 2 | RTE: 10.2 -> 6.0 | Helped |
| use WHAM with DPVO instead of SLAHMR | global trajectory recovery · EMDB 2 | Jitter: 31.3 -> 22.5 | Helped |
| use WHAM with DPVO instead of SLAHMR | global trajectory recovery · EMDB 2 | FS: 14.5 -> 4.4 | Helped |
| add feature integration to WHAM (ablation) | WHAM · EMDB 2 | MPJPE: 69.0 -> 59.3 | Helped |
| add AMASS 2D-to-3D lifting pretraining to WHAM (ablation) | WHAM · EMDB 2 | MPJPE: 83.0 -> 59.3 | Helped |
| add Neural Initialization to WHAM (ablation) | WHAM · EMDB 2 | MPJPE: 66.3 -> 59.3 | Helped |
| add camera angular velocity input to WHAM (ablation) | WHAM · EMDB 2 | RTE: 10.1 -> 6.0 | Helped |
| add trajectory refinement to WHAM (ablation) | WHAM · EMDB 2 | FS: 6.5 -> 4.4 | Helped |
Surprising findings
- Existing video-based 3D human pose methods are less accurate per frame than the best single-frame methods, which the authors attribute to training data limitations.
Limitations
- Learns human motion priors from AMASS, limiting generalization to out-of-distribution motions.
- Assumes the full body is within the field of view in its synthetic data generation.
Experimental setup
Datasets: AMASS 3DPW Human3.6M MPI-INF-3DHP InstaVariety BEDLAM
Benchmarks: 3DPW RICH EMDB
Metrics: MPJPE PA-MPJPE PVE Accel W-MPJPE100 WA-MPJPE100 RTE Jitter FS
Baselines: TCMR GLoT MPS-Net VIBE HMR2.0 ReFit TRACE SLAHMR GLAMR PACE
Builds on: CLIFF SPIN MotionBERT ProxyCap DPVO
Closest papers by meaning
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.89)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.87)
- BEDLAM2.0: Synthetic humans and cameras in motion (NeurIPS 2025 · similarity 0.87)
- Lifting Motion to the 3D World via 2D Diffusion (CVPR 2025 · similarity 0.87)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.86)
- RoHM: Robust Human Motion Reconstruction via Diffusion (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.