Research map / 3D vision / Human pose estimation
FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeCap, a calibration-free hybrid system fusing a single LiDAR with expandable moving cameras for global multi-person motion capture, using pose-aware cross-sensor matching and a coarse-to-fine sensor-expandable pose optimizer.
Problem
Capturing global multi-person human motion in open environments without calibration between a LiDAR and moving cameras, where existing LiDAR-only and camera-only methods are inaccurate and calibrated fusion is rigid.
Approach
Matches LiDAR and camera humans via body-pose similarity refined by PnP-based key point reprojection optimization over sequences, estimates an alignment matrix without calibration, then fuses 2D and 3D key points with a transformer-based optimizer that can accept additional cameras.
Key results
- On Human-M3, FreeCap reaches 55.45/68.52 J/V Err(PS) versus 57.81/71.27 for LiveHPS and 69.42/83.35 for WHAM.
- On FreeMotion, FreeCap reaches 53.31/65.50 J/V Err(PS) versus 59.30/73.12 for LiveHPS, and Ang Err 11.14 versus 13.10.
- The PCM module reaches 98.10% matching accuracy on Human-M3 at 159.49 FPS, versus 89.88% for an all-keypoints variant running at 0.08 FPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add matching refinement (ablation) | FreeCap · Human-M3 | J/V Err(PS): 59.01/ 73.07 -> 55.45/68.52 | Helped |
| add temporal interaction (ablation) | FreeCap · Human-M3 | J/V Err(PS): 59.55/73.78 -> 55.45/68.52 | Helped |
| add temporal interaction (ablation) | FreeCap · FreeMotion | J/V Err(PS): 55.48/68.09 -> 53.31/65.50 | Helped |
| add sensor interaction (ablation) | FreeCap · Human-M3 | J/V Err(PS): 58.33/72.16 -> 55.45/68.52 | Helped |
| add sensor interaction (ablation) | FreeCap · FreeMotion | J/V Err(PS): 59.84/73.40 -> 53.31/65.50 | Helped |
| use multimodal fusion instead of body-pose fusion (ablation) | FreeCap · Human-M3 | J/V Err(PS): 58.84/75.81 -> 55.45/68.52 | Helped |
| use multimodal fusion instead of body-pose fusion (ablation) | FreeCap · FreeMotion | J/V Err(PS): 56.97/70.48 -> 53.31/65.50 | Helped |
| use FreeCap fusion instead of calibration-based fusion (ablation) | FreeCap · Human-M3 | J/V Err(PS): 59.46/72.77 -> 55.45/68.52 | Helped |
| use FreeCap fusion instead of calibration-based fusion (ablation) | FreeCap · FreeMotion | J/V Err(PS): 58.57/71.21 -> 53.31/65.50 | Helped |
| use sequential pose and keypoints instead of matching variants (ablation) | FreeCap PCM · Human-M3 | matching accuracy: 89.88 -> 98.10; 77.07 -> 98.10; 81.59 -> 98.10; 85.87 -> 98.10; 96.01 -> 98.10 | Helped |
Surprising findings
- The calibration-free FreeCap outperforms the authors' own calibration-based LiDAR-camera fusion variant (Cali-based) on most metrics.
Limitations
- Human-M3 provides only merged LiDAR point clouds, so the single-LiDAR setting is not fully matched to the evaluation data.
- WHAM's global evaluation was omitted because it uses the first-frame location, making direct comparison unfair.
- Matching relies on body pose similarity, which the authors note may be ambiguous under symmetric anatomy or synchronized activities.
Experimental setup
Datasets: SURREAL Human-M3 FreeMotion
Benchmarks: Human-M3 FreeMotion
Metrics: J/V Err (PS) J/V Err (PST) Ang Err Accel Err SUCD matching accuracy FPS
Baselines: WHAM LiveHPS Cali-based fusion
Builds on: LiveHPS WHAM RTMPose SMPL
Closest papers by meaning
- LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment (CVPR 2024 · similarity 0.88)
- RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method (CVPR 2024 · similarity 0.84)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.84)
- Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera (CVPR 2024 · similarity 0.83)
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.83)
- MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.