Research map / 3D vision / Human pose estimation
PoseIRM: Enhance 3D Human Pose Estimation on Unseen Camera Settings via Invariant Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PoseIRM, a camera-parameter-free multi-view 3D human pose estimator trained with invariant risk minimization on synthetic 2D poses from randomly sampled camera settings, to improve generalization to unseen cameras.
Problem
Camera-parameter-free multi-view 3D human pose estimation generalizes poorly to unseen camera settings because existing datasets offer very few camera setups, so inferred camera parameters memorize the seen settings; naively synthesizing data for many new cameras is computationally prohibitive.
Approach
Synthesize 2D-3D pose pairs from many randomly sampled camera settings (via two pose synthesis algorithms modeling 2D detection error) and train the lifting network with an IRM-style variance penalty so that the model performs consistently across camera settings, letting it infer camera parameters from input rather than memorize them.
Key results
- On TotalCapture unseen cameras (2,4,6,8), PoseIRM reaches 14.3/25.5/18.0 MPJPE for W2/FS3/A3 vs. 30.7 average for MTF-Transformer baseline list (mean 16.6 vs 30.7 reported for MTF-Transformer in the mean column of seen/unseen subsets), with a seen-to-unseen drop of 3.8 vs 11.9 for the seen/unseen ga
- On Human3.6M, PoseIRM achieves 25.1 MPJPE (CPN, T=27), the best among camera-parameter-free methods (MTF-Transformer 26.2, FLEX 31.7 with CPN).
- Ablation on TotalCapture: MVCPSA reduces unseen-camera MPJPE from 27.2 (no PSA) to 16.6, and IRM (penalty P=1) lowers unseen MPJPE from 22.4 to 20.2 and seen MPJPE from 16.8 to 13.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PoseIRM instead of MTF-Transformer | MTF-Transformer (Res101) · TotalCapture seen cameras | MPJPE: 24.7 -> 14.7 | Helped |
| Apply PoseIRM instead of MTF-Transformer | MTF-Transformer (Res101) · TotalCapture unseen cameras | MPJPE: 30.7 -> 16.6 | Helped |
| Apply PoseIRM instead of MTF-Transformer | MTF-Transformer · Human3.6M | MPJPE: 26.2 -> 25.1 | Helped |
| Add pose synthesis augmentation (ablation) | PoseIRM · TotalCapture unseen cameras | MPJPE: 27.2 -> 18.7 | Helped |
| Use MVCPSA instead of DAPSA for pose synthesis (ablation) | PoseIRM · TotalCapture unseen cameras | MPJPE: 18.7 -> 16.6 | Helped |
| Use MVCPSA instead of DAPSA for pose synthesis (ablation) | PoseIRM · TotalCapture unseen cameras | P-MPJPE: 13.2 -> 12.6 | Helped |
| Add the IRM framework (ablation) | PoseIRM · TotalCapture seen cameras | MPJPE: 16.8 -> 13.2 | Helped |
| Add the IRM framework (ablation) | PoseIRM · TotalCapture unseen cameras | MPJPE: 22.4 -> 18.4 | Helped |
| Increase the IRM penalty from P=1 to P=100 (ablation) | PoseIRM · TotalCapture unseen cameras | MPJPE: 20.2 -> 18.4 | Helped |
| Increase the IRM penalty from P=1 to P=100 (ablation) | PoseIRM · TotalCapture seen cameras | MPJPE: 13.2 -> 14.7 | Hurt |
Surprising findings
- IRM improved performance on seen camera settings as well as unseen ones, which the authors attribute to better feature learning.
Limitations
- Generalization evaluated on only one extra camera setting split of TotalCapture; Human3.6M alone offers a single camera setting.
- Relies on a fixed 2D pose estimator (CPN or ResNet-101) and PoseFormer backbone; the IRM penalty weight must be tuned (P=1 vs P=100 trade-off).
Experimental setup
Datasets: Human3.6M TotalCapture
Benchmarks: Human3.6M TotalCapture
Metrics: MPJPE P-MPJPE
Baselines: MTF-Transformer Probabilistic Triangulation FLEX Canonical Fusion Crossview Fusion AdaFuse Learnable Triangulation PoseFormer MHFormer SRNet UGCN
Builds on: Invariant Risk Minimization PoseFormer MTF-Transformer
Closest papers by meaning
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.89)
- Learning Monocular 3D Human Pose Estimation From Multi-View Images (CVPR 2018 · similarity 0.89)
- MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation (CVPR 2022 · similarity 0.88)
- Generating Multiple Hypotheses for 3D Human Pose Estimation With Mixture Density Network (CVPR 2019 · similarity 0.88)
- Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.88)
- ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.