Research map / 3D vision / 3D generation
RoHM: Robust Human Motion Reconstruction via Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoHM, a diffusion-based framework that decomposes robust 3D human motion reconstruction from noisy, occluded monocular RGB(-D) video into global-trajectory (TrajNet) and local-pose (PoseNet) denoisers, linked by a TrajControl conditioning module and iterative inference.
Problem
Reconstructing globally coherent, plausible 3D human motion from monocular RGB(-D) videos under noise and occlusions, where regression methods lack global coherence and robustness and optimization-based priors are slow and brittle.
Approach
Decompose the problem into two conditional diffusion denoisers for root trajectory and local pose, couple them with a trainable zero-convolution control branch (TrajControl) cloned from frozen TrajNet encoder layers, run them iteratively at inference, and guide sampling with foot-skating and 2D reprojection scores.
Key results
- On AMASS Occ-L. at noise level 3, GMPJPE-occ improves from 88.0 mm (HuMoR) to 34.8 mm (RoHM), a reduction of over 48%.
- On PROX RGB-D, skating ratio drops to 0.038 versus 0.117 for HuMoR, and penetration distance to 3.36 mm versus 54.76 mm.
- RoHM is reported to be about 30 times faster than HuMoR at inference, excluding the initialization stage.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TrajControl to the iterative inference scheme (ablation) | RoHM · PROX RGB-D | Skating: 0.056 -> 0.038 | Helped |
| Add TrajControl to the iterative inference scheme (ablation) | RoHM · PROX RGB | Skating: 0.165 -> 0.116 | Helped |
| Add a second iterative pass without TrajControl (ablation) | RoHM · PROX RGB-D | Skating: 0.056 -> 0.046 | Helped |
| Add a second iterative pass without TrajControl (ablation) | RoHM · PROX RGB | Skating: 0.165 -> 0.146 | Helped |
| Add test-time score guidance (ablation) | RoHM without TrajControl · PROX RGB-D | Skating: 0.072 -> 0.056 | Helped |
| Add test-time score guidance (ablation) | RoHM without TrajControl · PROX RGB | Skating: 0.213 -> 0.165 | Helped |
| Use RoHM instead of HuMoR | PROX RGB-D · PROX RGB-D | Dist: 54.76 -> 3.36 | Helped |
| Use RoHM instead of HuMoR | EgoBody RGB · EgoBody RGB | Skating: 0.147 -> 0.010 | Helped |
Surprising findings
- Iterating TrajNet and PoseNet without the TrajControl module degrades results, indicating the control module is necessary for the iterative scheme.
Limitations
- Does not run online at real-time framerates.
- Focuses on full-body reconstruction; facial expressions and hand poses are not modeled.
- Does not incorporate 3D scene conditions for human-scene interaction.
Experimental setup
Datasets: AMASS PROX EgoBody
Benchmarks: AMASS PROX EgoBody
Metrics: MPJPE GMPJPE Accel Contact accuracy Skating ratio Ground penetration distance Acceleration norm
Baselines: VPoser-t HuMoR MDM++ CLIFF LEMO PhaseMP
Builds on: DDPM MDM ControlNet HuMoR SMPL-X
Closest papers by meaning
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.87)
- Coherent 3D Scene Diffusion From a Single RGB Image (NeurIPS 2024 · similarity 0.86)
- VideoMDM: Towards 3D Human Motion Generation From 2D Supervision (NeurIPS 2026 · similarity 0.86)
- WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion (CVPR 2024 · similarity 0.86)
- MAS: Multi-view Ancestral Sampling for 3D Motion Generation Using 2D Diffusion (CVPR 2024 · similarity 0.86)
- Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.