Research map / 3D vision / Human pose estimation
FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FreeMan, a large-scale multi-view 3D human pose dataset captured with 8 movable smartphones across 10 real-world scene types, with a semi-automated annotation pipeline and benchmarks for four pose estimation tasks.
Problem
Existing 3D human pose datasets are collected in controlled laboratory settings with fixed cameras and uniform backgrounds, so models trained on them generalize poorly to real-world conditions.
Approach
Collect multi-view RGB video with wirelessly NTP-synchronized, movable smartphone cameras in diverse real outdoor and indoor scenes, and annotate 3D poses semi-automatically (triangulation, SMPLify, and SAM/ControlNet-based erroneous-pose detection with human correction).
Key results
- HMR trained on FreeMan reaches 151.85mm MPJPE and 88.85mm PA-MPJPE on 3DPW, versus 118.54mm PA-MPJPE for Human3.6M-trained HMR (FreeMan-trained PARE gets 94.27mm MPJPE vs 118.31mm for Human3.6M-trained PARE in the 2D KPTs+SMPL setting, a 8.76% PA-MPJPE gain noted in the paper).
- Multi-view VoxelPose trained on Human3.6M fails on FreeMan (0% AP at thresholds below 100mm), while FreeMan-trained VoxelPose achieves Recall@500mm of 96.68% and MPJPE@500mm of 61.29mm on Human3.6M.
- HumanNeRF on 10 FreeMan scenes yields PSNR from 23.86 to 30.11 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train HMR on FreeMan instead of Human3.6M | HMR · 3DPW test | PA-MPJPE (mm): 133.13 -> 87.93 | Helped |
| Train HMR on FreeMan instead of HuMMan | HMR · 3DPW test | PA-MPJPE (mm): 192.75 -> 87.93 | Helped |
| Train PARE on FreeMan instead of HuMMan | PARE · 3DPW test | PA-MPJPE (mm): 63.11 -> 60.39 | Helped |
| Train SimpleBaseline on FreeMan instead of Human3.6M | SimpleBaseline · AIST++ | PA-MPJPE (mm): 138.98 -> 105.85 | Helped |
| Train SimpleBaseline on FreeMan instead of HuMMan | SimpleBaseline · AIST++ | MPJPE (mm): 255.5 -> 156.96 | Helped |
| Expand SimpleBaseline training from sampled FreeMan data to the full FreeMan training set | SimpleBaseline · FreeMan | MPJPE (mm): 90.53 -> 79.22 | Helped |
| Train MHFormer on FreeMan instead of Human3.6M | MHFormer · AIST++ | PA-MPJPE (mm): 133.37 -> 88.79 | Helped |
| Train VoxelPose on FreeMan instead of Human3.6M | VoxelPose · Human3.6M | MPJPE@500mm (mm): 103.02 -> 58.30 | Helped |
| Train VoxelPose on Human3.6M and test on FreeMan | VoxelPose · FreeMan | AP@100mm (%): 0.00 | Hurt |
Surprising findings
- Models trained on the Human3.6M or HuMMan datasets fail to localize humans on FreeMan in multi-view cross-domain tests, indicating a large domain gap.
Limitations
- Prompts for the generation model need careful tuning for quality.
- Error pose detection is limited by the accuracy of the human image generation model.
- Only 1% of HuMMan is publicly available, limiting comparison scope.
Experimental setup
Datasets: FreeMan Human3.6M HuMMan 3DPW AIST++
Benchmarks: FreeMan 3DPW Human3.6M AIST++
Metrics: MPJPE PA-MPJPE AP@25mm-100mm Recall@500mm MPJPE@500mm PSNR SSIM LPIPS
Baselines: HMR PARE SimpleBaseline VideoPose3D PoseFormer MHFormer VoxelPose HumanNeRF
Builds on: HMR PARE VoxelPose HumanNeRF SMPLify
Closest papers by meaning
- Benchmarking and Analyzing 3D Human Pose and Shape Estimation Beyond Algorithms (NeurIPS 2022 · similarity 0.90)
- Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation From Monocular Video (CVPR 2022 · similarity 0.89)
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.89)
- 3D Human Pose Estimation Using Spatio-Temporal Networks with Explicit Occlusion Training (AAAI 2020 · similarity 0.88)
- 3D Human Pose Perception from Egocentric Stereo Videos (CVPR 2024 · similarity 0.88)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.