Research map / 3D vision / Human pose estimation
Score-Guided Diffusion for 3D Human Recovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ScoreHMR, which refines off-the-shelf SMPL regression estimates by DDIM inversion plus score guidance from keypoint, multi-view, and temporal likelihood terms using a task-agnostic pose diffusion model.
Problem
Monocular and multi-view 3D human pose and shape recovery from image observations, where regression lacks image-model alignment and optimization-based fitting is slow and sensitive to initialization and local minima.
Approach
Train a conditional diffusion model on SMPL pose parameters given image features, then refine a regression estimate by DDIM inversion followed by deterministic guided DDIM sampling, where the guidance term (keypoint reprojection, cross-view consistency, or temporal smoothness) acts as the data term; the same trained prior is reused across tasks without retraining.
Key results
- On 3DPW keypoint fitting, HMR 2.0 + ScoreHMR-b reduces PA-MPJPE from 54.3 to 51.1 mm, beating ProHMR-fitting (55.1) and SMPLify (60.1).
- Multi-view refinement on Human3.6M lowers MPJPE from 52.8 (HMR 2.0) to 44.7 mm with ScoreHMR-b.
- Human motion refinement with HMR 2.0 gives a 21.3% (3DPW) and 40.5% (EMDB 1) relative acceleration error improvement over ProHMR-fitting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ScoreHMR-b to HMR 2.0 for keypoint fitting | HMR 2.0 · 3DPW (14) | PA-MPJPE (mm): 54.3 -> 51.1 | Helped |
| Apply ScoreHMR-b to HMR 2.0 for keypoint fitting | HMR 2.0 · EMDB 1 (24) | PA-MPJPE (mm): 78.7 -> 76.6 | Helped |
| Apply cross-view ScoreHMR refinement | HMR 2.0 · H36M (14) | MPJPE: 52.8 -> 44.7 | Helped |
| Apply cross-view ScoreHMR refinement | HMR 2.0 · Mannequin (17) | MPJPE: 156.0 -> 148.3 | Helped |
| Apply temporal ScoreHMR refinement | HMR 2.0 · 3DPW (14) | Acc Err: 17.3 -> 11.1 | Helped |
| Apply temporal ScoreHMR refinement | HMR 2.0 · EMDB 1 (24) | Acc Err: 23.7 -> 11.9 | Helped |
| Add shape parameters β to ScoreHMR (ablation) | ScoreHMR · single-image human mesh recovery | performance | No clear effect |
| Train image feature backbone end-to-end with the diffusion model (ablation) | ScoreHMR · single-frame model fitting | PA-MPJPE | No clear effect |
| Use PARE image features and EFT fits instead of ProHMR features and SPIN fits (ablation) | ScoreHMR · 3DPW (14) | PA-MPJPE (mm): 55.7 -> 54.7 | Helped |
| Use PARE image features and EFT fits instead of ProHMR features and SPIN fits (ablation) | ScoreHMR · EMDB 1 (24) | PA-MPJPE (mm): 77.8 -> 77.1 | Helped |
| Replace SPIN pseudo-ground-truth fits with EFT fits while using ProHMR features (ablation) | ScoreHMR · 3DPW (14) | PA-MPJPE (mm): 55.7 -> 55.5 | Helped |
| Replace SPIN pseudo-ground-truth fits with EFT fits while using ProHMR features (ablation) | ScoreHMR · EMDB 1 (24) | PA-MPJPE (mm): 77.8 -> 77.4 | Helped |
Surprising findings
- Including SMPL shape parameters in the diffusion model gave no performance improvement, which the authors attribute to shape being easier to infer from a single image.
- SMPLify applied on top of regression increased 3D pose errors (e.g., HMR 2.0 54.3 to 60.1 mm PA-MPJPE on 3DPW).
Limitations
- Only the pose parameters are modeled; shape parameters did not help and were left out of the main method.
- Relies on pseudo ground-truth SMPL annotations for training because paired image-SMPL labels are not generally available.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP COCO MPII 3DPW EMDB Mannequin Challenge
Benchmarks: 3DPW EMDB 1 Human3.6M Mannequin Challenge
Metrics: PA-MPJPE MPJPE Acceleration error
Baselines: HMR 2.0 ProHMR SMPLify ProHMR-fitting VIBE-opt LGD LFMM
Builds on: DDPM DDIM DPS (diffusion posterior sampling) ProHMR HMR 2.0
Closest papers by meaning
- ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization (ICML 2025 · similarity 0.91)
- ScoreHypo: Probabilistic Human Mesh Estimation with Hypothesis Scoring (CVPR 2024 · similarity 0.88)
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.88)
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.87)
- DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · similarity 0.86)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.