Research map / Image generation / Fast diffusion sampling
DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiSRT-In-Bed, a diffusion model over SMPL parameters conditioned on overhead depth images, trained on large synthetic depth data and fine-tuned on limited real data for in-bed human mesh recovery.
Problem
In-bed human mesh recovery from depth images suffers from scarce real-world labeled data and a large synthetic-to-real domain gap, causing poor generalization across coverings and environments.
Approach
Reformulates in-bed mesh recovery as conditional generative denoising of SMPL parameters given depth images, with decoupled synthetic pre-training followed by real-data fine-tuning using an adaptive learning rate schedule.
Key results
- Under Sim+11%Real data, DiSRT-In-Bed reaches 74.37 mm MPJPE vs 85.79 mm for BodyMAP on the SLP home test split.
- With Sim+100%Real data, MPJPE drops to 50.81 mm, versus 57.06 mm for BodyMAP.
- Trained on synthetic data alone, DiSRT-In-Bed (109.73 mm MPJPE) slightly underperforms BodyPressure (103.47 mm).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiSRT-In-Bed instead of BodyMAP with Sim+11%Real training data | In-bed mesh recovery · SLP home test, Sim+11%Real | MPJPE: 85.79 -> 74.37 | Helped |
| Use DiSRT-In-Bed instead of BodyMAP with Sim+100%Real training data | In-bed mesh recovery · SLP home test, Sim+100%Real | PVE: 69.95 -> 61.18 | Helped |
| Use DiSRT-In-Bed instead of BodyMAP | In-bed mesh recovery · SLP home test, uncover | MPJPE: 51.26 -> 46.01 | Helped |
| Add synthetic training data (ablation) | DiSRT-In-Bed · SLP home test, varying real-data utilization | MPJPE | Helped |
| Add the Sim-to-Real training strategy (ablation) | BodyMAP · SLP home test, varying real-data utilization | MPJPE | Helped |
| Use six rather than four downsampling blocks (ablation) | DiSRT-In-Bed · SLP home test, varying real-data utilization | MPJPE | Helped |
| Replace adaptive-normalization conditioning with in-context conditioning (ablation) | DiSRT-In-Bed · SLP home test, varying real-data utilization | MPJPE | Hurt |
| Add attention blocks in the final diffusion-model layers (ablation) | DiSRT-In-Bed · SLP home test, varying real-data utilization | MPJPE | Helped |
Surprising findings
- Trained only on synthetic data, the proposed model slightly underperforms BodyPressure, which the authors attribute to BodyPressure's separate shape-prediction network.
Limitations
- Synthetic-only performance is worse than BodyPressure
- Evaluation on hospital setting is qualitative only, without ground-truth meshes
- Restricted to resting poses and a single overhead depth camera setup
Experimental setup
Datasets: BodyPressureSD SLP
Benchmarks: SLP home setting (participants 81-102) SLP hospital setting (qualitative)
Metrics: MPJPE PVE 3D shape error
Baselines: HMR2.0 BodyPressure BodyMAP BodyMAP+S2R
Builds on: DDPM SMPL BodyPressure BodyMAP ADM diffusion U-Net
Closest papers by meaning
- PI-HMR: Towards Robust In-bed Temporal Human Shape Reconstruction with Contact Pressure Sensing (CVPR 2025 · similarity 0.87)
- ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization (ICML 2025 · similarity 0.87)
- DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · similarity 0.86)
- R-Cyclic Diffuser: Reductive and Cyclic Latent Diffusion for 3D Clothed Human Digitalization (CVPR 2024 · similarity 0.86)
- Score-Guided Diffusion for 3D Human Recovery (CVPR 2024 · similarity 0.85)
- BodyMAP - Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.