Research map / 3D vision / Human pose estimation
PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMPLify-IB, a gravity- and penetration-aware monocular optimizer that regenerates SMPL pseudo-labels for the TIP in-bed dataset, and PI-HMR, a temporal pressure-sequence network with multi-scale pressure/joint-guided feature fusion, KD pretraining from CLIFF, and VQ-VAE test-time optimization.
Problem
Reconstructing 3D human shape and motion in bed from pressure-sensing bedsheets is hampered by weak model designs for pressure data and by low-quality monocular SMPL pseudo-labels with depth ambiguity and self-penetration.
Approach
Treat pressure images as a distinct modality by fusing global, local, and sampling features guided by high-pressure TopK masks, 2D joint positions, and sensor spatial position embeddings; add gravity constraints and a lightweight self-penetration term to the optimizer that generates in-bed labels.
Key results
- PI-HMR reaches 59.46 mm MPJPE on TIP vs 76.47 mm for PI-Mesh (about 17 mm reduction) and 64.37 mm for TCMR.
- Adding cross-modal KD and TTO brings the best MPJPE to 55.50 mm (PI-HMR + KD + TTO).
- SMPLify-IB detection module is 53.9x faster than SMPLify-XMC per iteration (0.42 s vs 22.62 s) with 98.32% accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PI-HMR instead of PI-Mesh | TIP pressure-sequence HPS · TIP test | MPJPE: 76.47 -> 59.46 | Helped |
| Add cross-modal knowledge distillation | PI-HMR · TIP test | MPJPE: 59.46 -> 57.13 | Helped |
| Add test-time optimization | PI-HMR · TIP test | MPJPE: 59.46 -> 57.76 | Helped |
| Add both cross-modal knowledge distillation and test-time optimization | PI-HMR · TIP test | MPJPE: 59.46 -> 55.50 | Helped |
| Use all global, local, pressure-sampling, and joint-sampling feature branches (ablation) | PI-HMR + KD MFF · TIP test | MPJPE: 62.23 -> 57.13 | Helped |
| Add joint-guided sampling alongside pressure-guided sampling (ablation) | PI-HMR + KD MFF · TIP test | MPJPE: 59.03 -> 57.13 | Helped |
| Set TopK sampling to 128 (ablation) | PI-HMR + KD · TIP test | MPJPE: Top 8: 58.62; Top 32: 57.66; Top 128: 57.13; Top 256: 58.64 | Helped |
| Add learnable masks (ablation) | PI-HMR + KD MFF · TIP test | MPJPE: 60.95 -> 57.13 | Helped |
| Add spatial position embedding (ablation) | PI-HMR + KD MFF · TIP test | MPJPE: 60.65 -> 57.13 | Helped |
| Add AttentionPooling (ablation) | PI-HMR + KD MFF · TIP test | MPJPE: 59.21 -> 57.13 | Helped |
| Add CLIFF output distillation without feature distillation (ablation) | HMR cross-modal KD · TIP test | MPJPE: 75.06 -> 77.86 | Hurt |
| Add CLIFF feature distillation (ablation) | HMR cross-modal KD · TIP test | MPJPE: 75.06 -> 67.34 | Helped |
Surprising findings
- Cross-modal KD with output supervision from CLIFF alone degraded results (77.86 mm vs 75.06 mm baseline), and only feature-level KD helped.
Limitations
- Evaluated only on the TIP dataset with 9 subjects, limiting diversity.
- Gravity and self-penetration constraints are heuristic and only penalize limb distance to bed plane, which may worsen self-penetration.
- Pseudo-GT labels are not validated against true 3D ground truth; limb height serves only as a rough proxy.
Experimental setup
Datasets: TIP SLP
Benchmarks: TIP
Metrics: MPJPE PA-MPJPE MPVE ACC-ERR
Baselines: HMR HMR-KD BodyMap-WS TCMR MPS-NET PI-Mesh CLIFF SMPLify-XMC
Builds on: HMR TCMR CLIFF SMPLify PI-Mesh VQ-VAE
Closest papers by meaning
- BodyMAP - Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed (CVPR 2024 · similarity 0.88)
- DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery (CVPR 2025 · similarity 0.87)
- MotionPRO: Exploring the Role of Pressure in Human MoCap and Beyond (CVPR 2025 · similarity 0.83)
- UMotion: Uncertainty-driven Human Motion Estimation from Inertial and Ultra-wideband Units (CVPR 2025 · similarity 0.82)
- PostureHMR: Posture Transformation for 3D Human Mesh Recovery (CVPR 2024 · similarity 0.82)
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.