Research map / 3D vision / Human pose estimation
ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ProxyCap, a real-time monocular full-body motion capture method that learns human-centric world-space motion from a synthetic 2D-skeleton proxy dataset built on AMASS, refined by a contact-aware neural descent module.
Problem
Real-time monocular full-body motion capture that is accurate and physically plausible (foot-ground contact) in world space, especially with moving hand-held cameras, where camera-centric regression is ambiguous and SLAM/SfM-based optimization is slow.
Approach
Use 2D skeleton sequences as a proxy representation, synthesize unlimited proxy-to-motion pairs by rendering AMASS motions under random virtual camera trajectories, regress motion in a human-centric coordinate frame with relative camera poses, and refine with a learned contact-aware neural descent module that replaces explicit gradient-based optimization.
Key results
- On EgoBody, W-MPJPE 385.5 and WA-MPJPE 131.3 versus GLAMR's 416.1 and 239.0 (SLAM-based SLAHMR and PACE are better on WA-MPJPE/PA-MPJPE in some cases).
- Neural descent reduces W-MPJPE on Dynamic Human3.6M from 644.8 to 605.4 and PA-MPJPE from 48.4 to 45.9.
- Runs at 30 FPS on a laptop RTX4060 with about 60 fps initial estimation and 73 fps neural descent, an order of magnitude faster than prior methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the Neural Descent module (ablation) | ProxyCap · Dynamic Human3.6M | W-MPJPE: 644.8 -> 605.4 | Helped |
| Add the Neural Descent module (ablation) | ProxyCap · Dynamic Human3.6M | PA-MPJPE: 48.4 -> 45.9 | Helped |
| Use ProxyCap instead of GLAMR | Monocular world-space motion capture · EgoBody | W-MPJPE: 416.1 -> 385.5 | Helped |
| Use ProxyCap instead of GLAMR | Monocular world-space motion capture · RICH | PA-MPJPE: 79.9 -> 56.0 | Hurt |
Surprising findings
- On RICH and EgoBody, the method does not beat SLAM-based PACE and SLAHMR on several metrics such as PA-MPJPE and ACCEL, despite being SLAM-free.
Limitations
- Depth ambiguity from 2D joint observations, especially for nearly static poses.
- Assumes planar ground, so motions like climbing stairs or boardskating are not handled.
- SLAM-based methods (PACE, SLAHMR) remain more accurate on some metrics such as PA-MPJPE and ACCEL on RICH/EgoBody.
Experimental setup
Datasets: AMASS InterHand RICH EgoBody Dynamic Human3.6M Human3.6M 3DPW EHF
Benchmarks: RICH EgoBody Dynamic Human3.6M
Metrics: W-MPJPE WA-MPJPE PA-MPJPE ACCEL ground penetration foot floating
Baselines: GLAMR SLAHMR PACE PyMAF-X
Builds on: SMPL GLAMR AMASS HuMoR PyMAF-X
Closest papers by meaning
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.84)
- Lifting Motion to the 3D World via 2D Diffusion (CVPR 2025 · similarity 0.84)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.84)
- MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond (CVPR 2025 · similarity 0.84)
- Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos (NeurIPS 2024 · similarity 0.84)
- Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.