Research map / 3D vision / Human pose estimation
FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FactorizedHMR, a two-stage video human mesh recovery method that deterministically regresses a torso-root anchor and completes non-torso and world-motion articulation with masked conditional flow matching, trained partly on a Uni3C-based synthetic dataset.
Problem
Video human mesh recovery is ambiguous under occlusion and depth uncertainty; deterministic regressors average to biased solutions while fully probabilistic models are inefficient and unstable across all body variables.
Approach
Factorize HMR by uncertainty: a deterministic transformer estimates a torso/root/shape/camera anchor, and a masked flow-matching model generates only the ambiguous non-torso articulation and world-motion variables, with joint-rotation/position composite targets, joint-bone consistency, direct projection loss, and feature-aware classifier-free guidance.
Key results
- Camera-space: ours achieves best MPJPE on 3DPW (56.2 vs GVHMR 56.9), RICH (65.8), and EMDB (73.6) among compared methods, with PA-MPJPE competitive but not always best.
- 3DPW-XOCC occlusion benchmark: MPJPE 66.3 vs GENMO 76.2 and PARE+VIBE 97.3.
- World-space: W-MPJPE on EMDB improves to 192.5 (GENMO 210.9), but WA-MPJPE (70.5) trails GENMO (65.1) and jitter is worse than GENMO on RICH and EMDB.
Surprising findings
- Fully probabilistic torso/non-torso variant performs worse than fully deterministic on 3DPW (MPJPE 68.9 vs 60.2), indicating stochastic torso/root generation destabilizes estimates.
- Runtime is about 3.92 s per 1193-frame sequence versus 0.41 s for GVHMR, showing a substantial inference cost for the flow-matching stage.
Limitations
- Fixed uncertainty partition between Stage 1 and Stage 2; errors in the anchor propagate to Stage 2.
- Probabilistic refinement increases inference cost, limiting deployment.
- Benchmarks provide only single reference motion, so calibration and multi-hypothesis quality are not measured.
Experimental setup
Datasets: AMASS BEDLAM H36M 3DPW synthetic Uni3C videos
Benchmarks: 3DPW RICH EMDB 3DPW-XOCC
Metrics: MPJPE PA-MPJPE PVE WA-MPJPE W-MPJPE Root translation Jitter Foot-sliding
Baselines: GVHMR GENMO TRAM WHAM TRACE HMR2.0 CLIFF HybrIK VIBE PARE NIKI RoHM
Builds on: GVHMR flow matching WHAM Uni3C
Closest papers by meaning
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.89)
- Self-Supervised 3D Human Mesh Recovery from a Single Image with Uncertainty-Aware Learning (AAAI 2024 · similarity 0.89)
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.88)
- ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization (ICML 2025 · similarity 0.88)
- MeshPose: Unifying DensePose and 3D Body Mesh Reconstruction (CVPR 2024 · similarity 0.87)
- DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.