Research map / 3D vision / Human pose estimation
ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ManiPose, a deterministic manifold-constrained multi-hypothesis 2D-to-3D human pose lifting model trained with multiple choice learning, improving pose consistency (skeleton rigidity/symmetry) while staying competitive on MPJPE.
Problem
Monocular 3D human pose lifting from 2D keypoints is ill-posed due to depth ambiguity; regression models produce inconsistent skeletons (varying bone lengths, asymmetry) that standard metrics such as MPJPE, P-MPJPE and PCK do not capture.
Approach
Predict K pose hypotheses via multiple choice learning (resilient MCL with winner-takes-all and scoring losses) where each hypothesis is decoded from predicted segment lengths and 6D joint rotations with forward kinematics, so all hypotheses lie on the human pose manifold, without a generative model.
Key results
- On Human3.6M, ManiPose (K=5, oracle) reaches MPJPE 39.1 mm with MPSSE 0.3 and MPSCE 0.5, versus MixSTE 40.9 mm MPJPE with MPSCE 9.9.
- On MPI-INF-3DHP with ground-truth 2D poses, ManiPose oracle obtains PCK 98.4 and AUC 77.0, with MPSCE 1.3 versus 21.6 for MixSTE.
- Ablation: removing multiple hypotheses (K=1) with manifold constraint raises MPJPE to 44.6 mm while keeping consistency, showing both components are needed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multiple hypotheses (K=5) (ablation) | ManiPose with manifold constraint · Human3.6M | MPJPE: 44.6 -> 39.1 | Helped |
| add multiple hypotheses (K=5) (ablation) | ManiPose with manifold constraint · Human3.6M | MPSSE: 0.3 -> 0.3 | No clear effect |
| add multiple hypotheses (K=5) (ablation) | ManiPose with manifold constraint · Human3.6M | MPSCE: 0.5 -> 0.5 | No clear effect |
| replace manifold regularization with a manifold constraint (ablation) | single-hypothesis ManiPose with manifold regularization · Human3.6M | MPSSE: 5.7 -> 0.3 | Helped |
| replace manifold regularization with a manifold constraint (ablation) | single-hypothesis ManiPose with manifold regularization · Human3.6M | MPSCE: 7.3 -> 0.5 | Helped |
| replace manifold regularization with a manifold constraint (ablation) | single-hypothesis ManiPose with manifold regularization · Human3.6M | MPJPE: 42.3 -> 44.6 | Mixed |
| add manifold regularization (ablation) | MixSTE · Human3.6M | MPSSE: 8.8 -> 5.7 | Helped |
| add manifold regularization (ablation) | MixSTE · Human3.6M | MPSCE: 9.9 -> 7.3 | Helped |
| add the manifold constraint and multiple hypotheses instead of using unconstrained single-hypothesis regression (ablation) | MixSTE · Human3.6M | MPJPE: 40.9 -> 39.1 | Helped |
| add the manifold constraint and multiple hypotheses instead of using unconstrained single-hypothesis regression (ablation) | MixSTE · Human3.6M | MPSSE: 8.8 -> 0.3 | Helped |
| add the manifold constraint and multiple hypotheses instead of using unconstrained single-hypothesis regression (ablation) | MixSTE · Human3.6M | MPSCE: 9.9 -> 0.5 | Helped |
Surprising findings
- Standard regression models that minimize MPJPE/MSE are shown to produce inconsistent skeletons, and MPJPE improvements of MixSTE come with worse consistency.
- Simply adding a manifold regularization term to a single-hypothesis model improves consistency only partially compared with the manifold constraint.
Limitations
- Forward kinematics is inherently sequential across joints, which may slow computation.
- Imposing human body articulation limits is not addressed.
- Oracle evaluation uses the hypothesis closest to ground truth, so oracle metrics favor multi-hypothesis outputs; aggregated results are weaker on MPJPE.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP
Benchmarks: Human3.6M MPI-INF-3DHP
Metrics: MPJPE P-MPJPE MPSCE MPSSE PCK AUC oracle MPJPE
Baselines: ST-GCN VideoPose3D PoseFormer Anatomy3D MixSTE Wehrbein et al. GFPose D3DP Sharma et al. DiffPose MHFormer P-STMO
Builds on: MixSTE multiple choice learning resilient multiple choice learning winner-takes-all loss 6D rotation representation
Closest papers by meaning
- PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space (NeurIPS 2025 · similarity 0.90)
- HuManiFlow: Ancestor-Conditioned Normalising Flows on SO(3) Manifolds for Human Pose and Shape Distribution Estimation (CVPR 2023 · similarity 0.90)
- COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation (NeurIPS 2026 · similarity 0.88)
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.88)
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.88)
- PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.