Research map / 3D vision / Depth and stereo
Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Dyn-HaMR reconstructs 4D global two-hand motion from monocular videos of dynamic cameras via a three-stage optimization combining SLAM, a hand motion prior (HMP), and biomechanical/penetration constraints.
Problem
Recovering accurate global 4D trajectories and interacting two-hand meshes from monocular in-the-wild videos captured by moving (e.g., egocentric) cameras, where hand and camera motion are entangled and weak-perspective models fail.
Approach
A multi-stage optimization that initializes per-frame hand states from off-the-shelf trackers, composes them with SLAM (DPVO) camera motion and a learnable world scale factor, and refines with a neural-motion-field hand prior plus biomechanical and interpenetration losses.
Key results
- On H2O, Dyn-HaMR reduces G-MPJPE to 45.6 mm versus 96.9 mm for HaMeR and GA-MPJPE to 34.2 mm versus 75.7 mm.
- On HOI4D, Dyn-HaMR achieves G-MPJPE 58.5 mm and GA-MPJPE 45.6 mm, against HaMeR's 201.6 and 129.7 mm.
- On InterHand2.6M (30 fps, static camera), Dyn-HaMR attains MPJPE 7.94 mm versus 8.75 for ACR, the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Dyn-HaMR instead of ACR | InterHand2.6M 30 fps · InterHand2.6M 30 fps | MPJPE: 8.75 -> 7.94 | Helped |
| Use Dyn-HaMR instead of HaMeR | H2O · H2O | G-MPJPE: 96.9 -> 45.6 | Helped |
| Use Dyn-HaMR instead of HaMeR | HOI4D · HOI4D | G-MPJPE: 201.6 -> 58.5 | Helped |
| Add Stage III interaction optimization (ablation) | Dyn-HaMR on H2O · H2O | G-MPJPE: 51.9 -> 45.6 | Helped |
| Add biomechanical constraints (ablation) | Dyn-HaMR on H2O · H2O | G-MPJPE: 49.6 -> 45.6 | Helped |
| Add penetration constraints (ablation) | Dyn-HaMR on H2O · H2O | G-MPJPE: 46.3 -> 45.6 | Helped |
| Use generative infilling instead of simple interpolation (ablation) | Dyn-HaMR on H2O · H2O | G-MPJPE: 48.9 -> 45.6 | Helped |
| Add Stage II global optimization (ablation) | Dyn-HaMR on H2O · H2O | G-MPJPE: 84.5 -> 51.9 | Helped |
Surprising findings
- Adding DPVO camera compensation to HaMeR improves G-MPJPE on Ego-Exo4D and HOT3D but worsens jerk and PA-MPJPE is unchanged, showing naive SLAM is insufficient.
Limitations
- Works only for limited time horizons; long sequences remain unexplored.
- Relies on the single available hand motion prior (HMP) trained on Arctic.
- Does not model object interactions.
Experimental setup
Datasets: H2O HOI4D FPHA EgoDexter InterHand2.6M Ego-Exo4D HOT3D Arctic
Benchmarks: InterHand2.6M H2O HOI4D Ego-Exo4D HOT3D
Metrics: MPJPE MPVPE Acc Err G-MPJPE GA-MPJPE PA-MPJPE Jerk
Baselines: HaMeR ACR IntagHand HaMeR + DPVO InterWild DIR
Builds on: HaMeR HMP DPVO MANO NeMF
Closest papers by meaning
- HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos (CVPR 2025 · similarity 0.90)
- Reconstructing Hands in 3D with Transformers (CVPR 2024 · similarity 0.89)
- The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.88)
- StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video (NeurIPS 2026 · similarity 0.88)
- WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · similarity 0.88)
- EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wild (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.