Research map / 3D vision / Human pose estimation
MeshPose: Unifying DensePose and 3D Body Mesh Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MeshPose, which uses weak DensePose supervision via VertexPose heatmaps and lifts localized vertices to a low-poly 3D body mesh for real-time, pixel-accurate human mesh recovery.
Problem
Human mesh reconstruction (HMR) systems have high 2D reprojection error measured by DensePose localization metrics, while DensePose systems are pixel-accurate but provide no 3D mesh.
Approach
Predict 2D heatmaps for a sparse set of low-poly mesh vertices (VertexPose), supervise them weakly from DensePose UV annotations via barycentric cross-entropy and UV consistency losses, then lift visible vertices to 3D with a regressed depth and visibility-weighted fallback.
Key results
- On DensePose-COCO, MeshPose (HRNet32) reaches 47.87 AP and 57.62 AR, far above HMR baselines such as CLIFF (30.99 AP) and NIKI (25.11 AP).
- MeshPose (HRNet32) achieves 76.08 MPJPE and 46.73 PA-MPJPE on 3DPW... with 3D errors comparable to recent methods (e.g., Human3.6M MPJPE 92.70 vs. CLIFF 81.2 in table ordering noted in paper).
- MeshPoseXS runs at 124 FPS with about 21M parameters... ~10x fewer parameters than Metro while reaching better DensePose accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VertexPose instead of UV-regression DensePose (ablation) | DensePose models with matched backbones and training settings · DensePose-COCO | AP: 57.31 -> 58.87; 62.74 -> 63.32; 53.54 -> 54.08 | Mixed |
| use barycentric UV interpolation instead of the strongest-vertex UV (ablation) | VertexPose · DensePose-COCO | AP: 59.69 -> 61.24 | Helped |
| average UV over all vertices instead of using barycentric interpolation (ablation) | VertexPose · DensePose-COCO | AP: 61.24 -> 33.35 | Hurt |
| add 2D barycentric and UV consistency losses (ablation) | MeshPose with 3D losses · COCO-DensePose | AP: 38.22 -> 44.09 | Helped |
| add 2D barycentric and UV consistency losses (ablation) | MeshPose with 3D losses · 3DPW | PA-MPJPE: 45.31 -> 50.40 | Hurt |
| add visibility supervision (ablation) | MeshPose with 3D and 2D losses · COCO-DensePose | AP: 44.09 -> 45.51 | Helped |
| add visibility supervision (ablation) | MeshPose with 3D and 2D losses · 3DPW | PA-MPJPE: 50.40 -> 46.54 | Helped |
Surprising findings
- The VertexPose formulation with weak DensePose losses outperforms the UV-regression DensePose baseline when trained with identical data and settings.
Limitations
- Hand/arm flattening artifacts due to mesh upsampler lacking articulated-hand training data.
- Imperfect alignment under perspective distortion because of the weak perspective camera assumption.
- Hands and other fine details are not well captured in the low-poly mesh.
Experimental setup
Datasets: COCO DensePose-COCO Human3.6M MPI-INF-3DHP 3DPW
Benchmarks: DensePose-COCO COCO keypoints 3DPW Human3.6M
Metrics: DensePose AP DensePose AR AP-All AR-All AP-80% AR-80% MPJPE PA-MPJPE PVE FPS
Baselines: DaNet HybrIK PARE CLIFF PyMaf NIKI Metro Graphormer FastMetro PointHMR
Builds on: DensePose integral regression SMPL-style parametric mesh models HybrIK
Closest papers by meaning
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.87)
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.86)
- Self-Supervised 3D Human Mesh Recovery from a Single Image with Uncertainty-Aware Learning (AAAI 2024 · similarity 0.86)
- BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation (CVPR 2025 · similarity 0.86)
- Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation (NeurIPS 2024 · similarity 0.85)
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.