Research map / 3D vision / Human pose estimation
Self-Supervised 3D Human Mesh Recovery from a Single Image with Uncertainty-Aware Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Self-supervised monocular 3D human mesh recovery that regresses SMPL parameters from 2D joint heatmaps and depth maps, using self-consistency losses weighted by a learned body-joint uncertainty module.
Problem
Monocular 3D human mesh recovery normally requires large-scale 2D/3D ground-truth labels, which limits use on unlabeled in-the-wild images.
Approach
Uses 2D joints and estimated depth maps (trimmed with IQR and sampled along bones) as proxy inputs and pseudo-targets, and learns per-joint reliability from bone-length and depth-discrepancy symmetry consistency to weight the self-consistency losses.
Key results
- On 3DPW, the method reaches 159.4 MPJPE and 89.5 PA-MPJPE, beating Kundu et al. (2020) by 27.7 and 13.2 mm among self-supervised methods.
- On Human3.6M (Protocol 2), it achieves 85.4 PA-MPJPE, improving on Kundu et al. (2020) at 90.5.
- Ablation on 3DPW shows removing uncertainty raises MPJPE from 159.4 to 166.9, and removing depth input raises it to 165.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add uncertainty-weighted depth loss (ablation) | Ours · 3DPW | MPJPE: 166.9 -> 159.4 | Helped |
| add uncertainty-weighted depth loss (ablation) | Ours · 3DPW | PA-MPJPE: 93.4 -> 89.5 | Helped |
| add IQR depth trimming (ablation) | Ours without uncertainty weighting · 3DPW | MPJPE: 208.2 -> 166.9 | Helped |
| add IQR depth trimming (ablation) | Ours without uncertainty weighting · 3DPW | PA-MPJPE: 113.5 -> 93.4 | Helped |
| use depth input alongside 2D joints (ablation) | Ours · 3DPW | MPJPE: 165.9 -> 159.4 | Helped |
| use depth input alongside 2D joints (ablation) | Ours · 3DPW | PA-MPJPE: 97.6 -> 89.5 | Helped |
| use uncertainty weighting and depth-point sampling (ablation) | Ours · 3DPW | MPJPE: 182.6 -> 159.4 | Helped |
| use uncertainty weighting and depth-point sampling (ablation) | Ours · 3DPW | PA-MPJPE: 99.2 -> 89.5 | Helped |
| use Ours instead of Kundu et al. (2020) | 3D human mesh recovery · 3DPW | MPJPE: 187.1 -> 159.4; decreased by 27.7 | Helped |
| use Ours instead of Kundu et al. (2020) | 3D human mesh recovery · 3DPW | PA-MPJPE: 102.7 -> 89.5; decreased by 13.2 | Helped |
| use Ours instead of Kundu et al. (2020) | 3D human mesh recovery · Human3.6M Protocol 2 | PA-MPJPE: 90.5 -> 85.4; decreased by 5.1 | Helped |
Surprising findings
- Without IQR depth trimming, the depth-based variant performed even worse than not using depth for self-supervision.
Limitations
- Fails under severe occlusion where intermediate 2D joints and depth are unreliable.
- Reconstruction of feet and hands is weaker than supervised methods because the sparse 2D joint representation lacks those joints.
- Still lags fully supervised methods such as SPIN and PyMAF in accuracy.
Experimental setup
Datasets: Human3.6M 3DPW UP-3D
Benchmarks: 3DPW Human3.6M
Metrics: MPJPE PA-MPJPE
Baselines: HMR SPIN PyMAF SMPLify Mugaludi et al. 2021 Kundu et al. 2020 Tung et al. 2017a Rhodin et al. 2018
Builds on: HMR SMPL Kundu et al. 2020 self-supervised FG appearance consistency Jafarian and Park 2021 unsupervised depth estimation
Closest papers by meaning
- End-to-End Recovery of Human Shape and Pose (CVPR 2018 · similarity 0.90)
- Geometry-Driven Self-Supervised Method for 3D Human Pose Estimation (AAAI 2020 · similarity 0.90)
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.89)
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.89)
- Reconstructing Humans with a Biomechanically Accurate Skeleton (CVPR 2025 · similarity 0.88)
- BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.