Research map / 3D vision / Human pose estimation
ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ADHMR, which trains a learned HMR-Scorer to rank diffusion-generated mesh predictions, builds an automatic winner/loser preference dataset from them, and finetunes a diffusion HMR model (HypoNet) with DPO; the scorer is also used to clean pseudo-labeled training data.
Problem
Single-image human mesh recovery is ill-posed; probabilistic diffusion-based HMR methods often produce predictions misaligned with 2D image evidence and generalize poorly to in-the-wild images, and pseudo-labeled in-the-wild data is noisy.
Approach
Use a ViT-based reward model (HMR-Scorer) that takes UVD keypoints and pixel-aligned multi-scale image features to score mesh predictions, generate preference pairs from the base model's own samples without human labels, and align the diffusion denoiser with DPO; the same scorer filters noisy training data.
Key results
- ADHMR (ITW) with M=200 reaches 60.5 PVE / 52.6 MPJPE / 30.8 PA-MPJPE on 3DPW, versus 71.9 / 61.8 / 36.1 for the HypoNet base model at M=200.
- ADHMR (ITW) with only M=10 predictions (71.3 PVE, 61.3 MPJPE) outperforms the base HypoNet at M=200 on 3DPW MPJPE (61.8).
- Data cleaning with HMR-Scorer (threshold 0.6) improves OSX on 3DPW MPJPE from 100.4 to 99.4 and SMPLer-X on Human3.6M MPJPE from 59.8 to 57.5 while using less training data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| DPO-finetune HypoNet on the target benchmark | HypoNet · 3DPW, M=100 | PVE: 68.0 -> 65.4 | Helped |
| DPO-finetune HypoNet on the target benchmark | HypoNet · 3DPW, M=100 | MPJPE: 59.4 -> 57.2 | Helped |
| DPO-finetune HypoNet on the target benchmark | HypoNet · 3DPW, M=100 | PA-MPJPE: 35.9 -> 33.5 | Helped |
| Use DPO preference training instead of supervised pseudo-label finetuning | HypoNet · 3DPW, in-the-wild finetuning, M=100 | PVE: 70.2 -> 62.6 | Helped |
| Use DPO preference training instead of supervised pseudo-label finetuning | HypoNet · 3DPW, in-the-wild finetuning, M=100 | MPJPE: 61.3 -> 54.2 | Helped |
| Use DPO preference training instead of supervised pseudo-label finetuning | HypoNet · 3DPW, in-the-wild finetuning, M=100 | PA-MPJPE: 36.5 -> 32.0 | Helped |
| Add UVD coordinates and multi-scale image features to the scorer (ablation) | HMR-Scorer · GTA-Human | PLCC (PVE): 0.59 -> 0.63 | Helped |
| Represent joints with UVD coordinates instead of pose rotation vectors (ablation) | HMR-Scorer · GTA-Human | PLCC (PVE): 0.30 -> 0.63 | Helped |
| Add HMR-Scorer data cleaning | Hand4Whole · 3DPW | MPJPE: 115.2 -> 112.3 | Helped |
| Add HMR-Scorer data cleaning | OSX · Human3.6M | MPJPE: 69.5 -> 65.7 | Helped |
| Finetune on scorer training datasets with supervised learning (ablation) | HypoNet · 3DPW, M=200 | PVE: 71.9 -> 69.9 | Helped |
Surprising findings
- Supervised finetuning on the scorer's training datasets yields insignificant gains, suggesting the DPO improvement is not simply due to extra data.
- Adding HMR-Scorer-based data cleaning to Hand4Whole and OSX improves accuracy even though these models are trained on a smaller subset.
Limitations
- Scorer evaluation uses a test set constructed by perturbing ground-truth poses with Gaussian noise, not real HMR model errors.
- No existing benchmark for HMR score prediction, so comparisons rely on self-constructed test sets.
- Compute and training hyperparameters (e.g., beta, number of pairs K, training duration) are not detailed in the main text.
Experimental setup
Datasets: HI4D BEDLAM DNA-Rendering GTA-Human SPEC 3DPW Human3.6M InstaVariety MSCOCO MPII MPI-INF-3DHP EHF
Benchmarks: 3DPW Human3.6M EHF GTA-Human DNA-Rendering
Metrics: PVE MPJPE PA-MPJPE PA-PVE PLCC SRCC
Baselines: ScoreHypo ProHMR HuManiFlow HMDiff ScoreHMR HybrIK Zolly HMR 2.0 Hand4Whole OSX SMPLer-X ScoreNet
Builds on: Diffusion-DPO HypoNet ScoreHypo RankNet SMPL
Closest papers by meaning
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.91)
- Score-Guided Diffusion for 3D Human Recovery (CVPR 2024 · similarity 0.91)
- Incorporating Test-Time Optimization into Training with Dual Networks for Human Mesh Recovery (NeurIPS 2024 · similarity 0.90)
- DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · similarity 0.89)
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.88)
- ScoreHypo: Probabilistic Human Mesh Estimation with Hypothesis Scoring (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.