Research map / 3D vision / Human pose estimation
Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PiMForce, which fuses 3D hand posture (via a 3D ResNet34 over heatmaps) with forearm sEMG spectrograms to estimate 9-region hand pressure, and releases a 21-participant multimodal pose/sEMG/pressure dataset.
Problem
Estimating whole-hand pressure non-invasively is hard because sEMG alone confuses similar muscle activations across different hand postures, while vision methods fail under occlusion and require plane surfaces or line-of-sight views.
Approach
Fuse 3D hand pose (rendered as 3D heatmap volumes and encoded by a 3D ResNet34) with STFT-based sEMG features in a joint classification-regression network, so posture disambiguates sEMG patterns; the pose can come from an off-the-shelf RGB detector at inference time.
Key results
- PiMForce reaches 88.86% R^2, 6.65% NRMSE and 83.17% accuracy in within-dataset evaluation, versus 83.49% R^2 and 77.83% accuracy for sEMG-only.
- In cross-user evaluation, PiMForce achieves 70.06% R^2 and 72.01% accuracy versus 47.90% R^2 and 57.40% accuracy for sEMG-only.
- On plane and pinch postures with an off-the-shelf pose detector, PiMForce reaches 66.71% R^2 and 82.20% accuracy versus 40.30% R^2 and 67.90% accuracy for PressureVision++.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 3D hand posture information to sEMG | PiMForce · Evaluation | R²: 83.49 ± 16.40% -> 88.86 ± 11.92% | Helped |
| add 3D hand posture information to sEMG | PiMForce · Evaluation | NRMSE: 8.07 ± 2.62% -> 6.65 ± 2.11% | Helped |
| add 3D hand posture information to sEMG | PiMForce · Evaluation | Accuracy: 77.83 ± 11.56% -> 83.17 ± 9.38% | Helped |
| replace 3D hand-pose representation with hand angles (ablation) | PiMForce · Evaluation | R²: 88.86 ± 11.92% -> 84.22 ± 17.11% | Hurt |
| replace 3D hand-pose representation with hand angles (ablation) | PiMForce · Evaluation | NRMSE: 6.65 ± 2.11% -> 7.89 ± 2.61% | Hurt |
| replace 3D hand-pose representation with hand angles (ablation) | PiMForce · Evaluation | Accuracy: 83.17 ± 9.38% -> 78.22 ± 10.57% | Hurt |
| remove sEMG input (ablation) | PiMForce · Evaluation | R²: 88.86 ± 11.92% -> 66.32 ± 37.01% | Hurt |
| remove sEMG input (ablation) | PiMForce · Evaluation | NRMSE: 6.65 ± 2.11% -> 11.57 ± 3.95 | Hurt |
| remove sEMG input (ablation) | PiMForce · Evaluation | Accuracy: 83.17 ± 9.38% -> 70.08 ± 13.09% | Hurt |
| add 3D hand posture information to sEMG | PiMForce · Cross-user, whole interaction and posture | R²: 47.90 ± 9.97% -> 70.06 ± 4.02% | Helped |
| add 3D hand posture information to sEMG | PiMForce · Cross-user, whole interaction and posture | NRMSE: 14.14 ± 2.41% -> 10.70 ± 1.43% | Helped |
| add 3D hand posture information to sEMG | PiMForce · Cross-user, plane interaction and pinch posture | Accuracy: 66.00 ± 5.84% -> 82.20 ± 2.42% | Helped |
Surprising findings
- Adding joint angles instead of 3D pose to sEMG gives only marginal gains (under 0.5 percentage points on all metrics).
- PressureVision++ was non-functional in the authors' environment when using the original PressureVision code, due to poor generalization.
Limitations
- Accuracy drops on complex grasps and on Palm-Press (68.42% classification accuracy).
- Muscle fatigue effects are explicitly left outside the scope of the study.
- Inference without the glove depends on the accuracy of off-the-shelf hand pose detectors.
Experimental setup
Datasets: PiMForce multimodal dataset (self-collected)
Benchmarks: PiMForce dataset
Metrics: R^2 NRMSE MAE classification accuracy
Baselines: sEMG Only model 3D Hand Posture Only model sEMG + Hand Angles model PressureVision++
Builds on: PoseConv3D ResNet PressureVision++ MANO
Closest papers by meaning
- emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation (NeurIPS 2024 · similarity 0.89)
- emg2qwerty: A Large Dataset with Baselines for Touch Typing using Surface Electromyography (NeurIPS 2024 · similarity 0.86)
- EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision (CVPR 2025 · similarity 0.85)
- SpGesture: Source-Free Domain-adaptive sEMG-based Gesture Recognition with Jaccard Attentive Spiking Neural Network (NeurIPS 2024 · similarity 0.84)
- MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints (CVPR 2024 · similarity 0.83)
- Learning Dense Hand Contact Estimation from Imbalanced Data (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.