Research map / 3D vision / Human pose estimation
MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MotionPRO, a 12.4M-frame multi-view RGB, whole-body pressure mat, and optical MoCap dataset of 70 subjects and 400 motions, and proposes FRAPPE, which fuses pressure and RGB via cross-attention for physically plausible pose and global trajectory estimation.
Problem
Human motion capture from RGB focuses on visual alignment and ignores physical contact with the ground, causing jitter, drift, sliding, penetration and poor global trajectories that hurt virtual humans and humanoid robots. Existing pressure datasets are small or limited to lying or specific sports.
Approach
Use a whole-body pressure mat as a physical-interaction signal: a pressure-only temporal network with small-kernel ResNet encoder and long-short-term attention, and an RGB+pressure fusion (FRAPPE) where pressure queries image features in cross-attention, with orthographic 2D loss and contact loss.
Key results
- FRAPPE reaches 41.8 MPJPE and 58.6 PVE on MotionPRO, versus 54.7 MPJPE for CLIFF and 59.7 for VIBE in global pose evaluation.
- On global trajectory, FRAPPE achieves WMPJPE 60.8, WAMPJPE 44.6, RTE 41.6, Jitter 6.0 and WBCE 110.2, outperforming WHAM (75.6 WMPJPE, RTE 1023) and TRACE.
- Pressure-only estimation gives MPJPE 90.6 versus 133.0 for Intelligent Carpet fine-tuned on MotionPRO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the pressure-only IC model on MotionPRO | IC · MotionPRO | MPJPE: 299.3 -> 133.0 | Helped |
| use the proposed pressure-only pose and trajectory estimator instead of IC[FT] | IC[FT] · MotionPRO | MPJPE: 133.0 -> 90.6 | Helped |
| use the proposed pressure-only estimator instead of IC[FT] | IC[FT] · MotionPRO | GTraj: 80.5 -> 85.9 | Hurt |
| fuse pressure and RGB with FRAPPE | RGB-based pose estimation methods · MotionPRO | MPJPE: 56.4 -> 41.8 | Helped |
| add LSAM (ablation) | FRAPPE · MotionPRO | GTraj: 93.1 -> 62.2 | Helped |
| add FCAM (ablation) | FRAPPE · MotionPRO | GTraj: 66.0 -> 62.2 | Helped |
| add contact loss (ablation) | FRAPPE · MotionPRO | GTraj: 68.7 -> 62.2 | Helped |
| add 2D loss (ablation) | FRAPPE · MotionPRO | GTraj: 82.1 -> 62.2 | Helped |
Surprising findings
- Removing the FCAM fusion module yields better per-joint MPJPE (39.5) than the full FRAPPE (40.5), though the full model is better on global trajectory and global MPJPE.
- Removing the 2D orthographic loss improves jitter and foot sliding, indicating a trade-off between 2D alignment and physical plausibility.
Limitations
- Pressure-only results show a slight loss in global trajectory accuracy relative to the Intelligent Carpet fine-tuned baseline (GTraj 85.9 vs 80.5).
- Humanoid robot evaluation is partly in simulation (Webots) and real-world demonstration is qualitative only.
- Manual temporal synchronization between pressure, RGB and optical sensors.
Experimental setup
Datasets: MotionPRO
Benchmarks: MotionPRO
Metrics: MPJPE PMPJPE PVE Accel GTraj GMPJPE WMPJPE WAMPJPE RTE Jitter
Baselines: Intelligent Carpet VIBE CLIFF SMPLer-X TRACE WHAM PhysPT
Builds on: VIBE CLIFF WHAM Intelligent Carpet HRNet
Closest papers by meaning
- MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors (CVPR 2024 · similarity 0.89)
- Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos (NeurIPS 2024 · similarity 0.87)
- Capturing the Unseen: Vision-Free Facial Motion Capture Using Inertial Measurement Units (AAAI 2025 · similarity 0.86)
- EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision (CVPR 2025 · similarity 0.86)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.86)
- Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.