Research map / 3D vision / Human pose estimation
MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MMVP, a synchronized RGBD and plantar-pressure motion capture dataset with dense foot-contact labels, plus an RGBD-P SMPL fitting method and a monocular baseline VP-MoCap that predicts foot pressure/contact to improve global translation.
Problem
Existing mocap datasets annotate foot contact either via visual distance thresholding (low accuracy) or with pressure only for slow, small-range motion; there is no vision-pressure dataset for large-range, fast human motion with accurate dense foot contact.
Approach
Use high-precision insole pressure, normalized by subject body weight, to derive accurate dense foot-contact labels on SMPL foot vertices, use these in RGBD-P SMPL fitting, and predict pressure/contact from 2D keypoint sequences (FPP-Net) to guide monocular pose and translation optimization.
Key results
- RGBD-P SMPL fitting reaches F1 0.518 and IOU 0.507 for contact vs. PROX's 0.475 and 0.439 on MMVP.
- On MMVP-test, the proposed contact predictor reaches F1 0.532 and IOU 0.522, above fine-tuned BSTRO (0.496 F1).
- VP-MoCap reduces translation (Traj) error to 129.3 versus 211.4 for CLIFF and reduces PVE (Feet) to 91.9 versus 105.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RGBD and pressure signals for SMPL fitting | RGBD-P SMPL fitting · MMVP | F1: 0.475 -> 0.518 | Helped |
| Use RGBD and pressure signals for SMPL fitting | RGBD-P SMPL fitting · MMVP | MFCE: 9.348 -> 7.968 | Helped |
| Predict foot contact from temporal keypoints and optimize pose | VP-MoCap · MMVP-test | F1: 0.496 -> 0.532 | Helped |
| Use contact-guided pose and translation optimization | VP-MoCap · MMVP | Traj: 211.4 -> 129.3 | Helped |
| Add temporal foot consistency loss Lt (ablation) | VP-MoCap · MMVP | MPJPE: 87.0 -> 83.0 | Helped |
| Add temporal foot consistency loss Lt (ablation) | VP-MoCap · MMVP | Traj: 133.4 -> 129.3 | Helped |
| Add 3D contact constraint L3d and temporal foot consistency loss Lt (ablation) | VP-MoCap · MMVP | PVE (Feet): 98.9 -> 91.9 | Helped |
| Add 3D contact constraint L3d and temporal foot consistency loss Lt (ablation) | VP-MoCap · MMVP | Traj: 184.1 -> 129.3 | Helped |
| Add 3D contact constraint L3d and temporal foot consistency loss Lt (ablation) | VP-MoCap · MMVP | PVE: 108.2 -> 110.6 | Mixed |
Surprising findings
- Off-the-shelf contact predictors BSTRO and DECO, trained on visually annotated contacts, predict feet in contact for most frames, giving very high recall but poor precision.
- Removing the 3D foot-ground loss slightly lowers pose accuracy metrics but markedly worsens foot and translation errors, indicating a pose-translation trade-off.
Limitations
- Method is designed for static viewpoints and cannot handle moving cameras.
- The dataset has relatively homogeneous scenes, limiting scene generalization.
- Pose accuracy is slightly traded off against translation accuracy when 3D foot constraints are used.
Experimental setup
Datasets: MMVP
Benchmarks: MMVP MMVP-test
Metrics: E3d MFCE F1 IOU precision recall MPJPE PMPJPE PVE PVE (Feet)
Baselines: PROX LEMO BSTRO BSTRO[FT] DECO CLIFF TRACE SMPLer-X
Builds on: SMPLify PROX CLIFF BSTRO PhysCap
Closest papers by meaning
- MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond (CVPR 2025 · similarity 0.89)
- HUMAPS-4D: A Multimodal Dataset for HUman Motion Analysis with Physiological and Semantic informations (CVPR 2026 · similarity 0.88)
- Pressure2Motion: Hierarchical Human Motion Reconstruction from Ground Pressure with Text Guidance (CVPR 2026 · similarity 0.86)
- EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision (CVPR 2025 · similarity 0.83)
- RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method (CVPR 2024 · similarity 0.83)
- MoGenTS: Motion Generation based on Spatial-Temporal Joint Modeling (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.