Research map / 3D vision / Human pose estimation
Reconstructing Humans with a Biomechanically Accurate Skeleton
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a ViT transformer to regress biomechanical SKEL parameters from a single image, using SMPL-to-SKEL pseudo labels refined iteratively with a SKELify fitting step.
Problem
Single-image 3D human mesh recovery methods regress SMPL-style parameters with ball joints, producing unnatural joint rotations incompatible with biomechanics; no image dataset has SKEL ground truth.
Approach
Convert existing SMPL pseudo ground truth into SKEL parameters via optimization, then iteratively refine these pseudo labels during training with a keypoint-based fitting (SKELify) that includes joint-limit penalties, while regressing rotation matrices instead of Euler angles.
Key results
- On MOYO, HSMR reduces MPJPE from 123.3mm (HMR2.0) to 104.5mm and PA-MPJPE from 90.4mm to 79.6mm.
- HSMR matches HMR2.0 on COCO, LSP-Extended, PoseTrack, 3DPW and Human3.6M within about 0.5mm MPJPE difference.
- HMR2.0 violates 10-degree elbow/knee limits in 44-56% of MOYO frames, while HSMR has 0.0% violations for elbows and 0.0-0.5% for knees beyond 10 degrees (elbow 0% at 10 deg).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SKEL-based HSMR instead of HMR2.0 for reconstruction | Human mesh recovery · MOYO | MPJPE: 123.3 -> 104.5-18.8 | Helped |
| Use SKEL-based HSMR instead of HMR2.0 for reconstruction | Human mesh recovery · MOYO | PA-MPJPE: 90.4 -> 79.6-10.8 | Helped |
| Use SKEL-based HSMR instead of HMR2.0 for reconstruction | Human mesh recovery · 3DPW | MPJPE: 81.3 -> 81.5+0.2 | Hurt |
| Use SKEL-based HSMR instead of HMR2.0 for reconstruction | Human mesh recovery · Human3.6M | MPJPE: 50.0 -> 50.4+0.4 | Hurt |
| Use SKEL-based HSMR instead of HMR2.0 for reconstruction | Human mesh recovery · MOYO | MPVPE: 142.2 -> 120.1 | Helped |
| Use SKEL biomechanical constraints instead of SMPL regression | HMR2.0 · MOYO | Frequency of violation > 10°: 45.7% -> 3.9% | Helped |
| Regress Euler angles instead of continuous rotation representation (ablation) | HSMR (ViT-B) · MOYO | MPJPE: 124.0 -> 137.1 | Hurt |
| Add pseudo ground-truth refinement during training (ablation) | HSMR (ViT-B) · MOYO | MPJPE: 126.5 -> 124.0 | Helped |
Surprising findings
- Converting HMR2.0 outputs to SKEL via post-hoc fitting degrades accuracy on several datasets (e.g., MOYO MPJPE 130.5mm vs 123.3mm), so end-to-end regression is better than two-stage fitting.
Limitations
- Relies exclusively on pseudo ground truth for training, with no precise 3D labels.
- Temporal reconstructions show inevitable jitter.
- Fails under motion blur, extreme poses and rare viewpoints.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP COCO MPII AI Challenger AVA InstaVariety 3DPW MOYO LSP-Extended PoseTrack
Benchmarks: COCO LSP-Extended PoseTrack 3DPW Human3.6M MOYO
Metrics: PCK@0.05 PCK@0.1 MPJPE PA-MPJPE MPVPE PA-MPVPE joint limit violation frequency
Baselines: HMR2.0 PARE CLIFF HybrIK PLIKS HMR2.0 + SKEL fit
Builds on: SKEL HMR2.0 SPIN SMPLify
Closest papers by meaning
- REACTO: Reconstructing Articulated Objects from a Single Video (CVPR 2024 · similarity 0.88)
- Self-Supervised 3D Human Mesh Recovery from a Single Image with Uncertainty-Aware Learning (AAAI 2024 · similarity 0.88)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.87)
- GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data (NeurIPS 2025 · similarity 0.87)
- Toward Approaches to Scalability in 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.87)
- BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.