Research map / 3D vision / Human pose estimation
HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HaWoR, which reconstructs world-space egocentric hand motion by combining a camera-frame transformer hand estimator (with temporal attention modules), an adaptive DROID-SLAM with metric-scale alignment for camera trajectory, and a transformer motion infiller for out-of-view frames.
Problem
Reconstructing 3D hand motion in world coordinates from egocentric videos is hard because hands and camera move simultaneously, hands leave the field of view or are occluded, and monocular SLAM gives only up-to-scale camera trajectories.
Approach
Decouple world-space hand motion into camera-frame hand reconstruction plus world-space camera trajectory: hand-masked DROID-SLAM with adaptive metric scale estimation, and a transformer infiller that fills frames where hands are out of view, operating in a canonical space.
Key results
- On HOT3D world-space evaluation, HaWoR reaches W-MPJPE 33.20 and WA-MPJPE 11.27 versus 119.41 and 39.46 for the best baseline HMP-SLAM.
- Camera trajectory ATE-S drops to 9.31 mm versus 14.28 for DROID + Metric3DV2.
- On DexYCB under 75%-100% occlusion, PA-MPJPE is 5.07 versus 5.68 for WiLoR, with single-forward-pass runtime of 40 ms per frame, 75% faster than HMP-SLAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HaWoR's temporal hand-motion estimation to WiLoR | WiLoR hand reconstruction · DexYCB test, 75%-100% occlusion | PA-MPJPE: 5.68 -> 5.07 | Helped |
| Use adaptive egocentric SLAM and AdaSM | DROID-SLAM with Metric3DV2 on HOT3D · HOT3D, ATE-S | ATE-S: 21.07 -> 14.61 | Helped |
| Add pretrained ViT (ablation) | HaWoR · HOT3D | PA-MPJPE: 7.59 -> 4.79 | Helped |
| Add IAM and PAM (ablation) | HaWoR · HOT3D | W-MPJPE: 44.60 -> 33.20 | Helped |
| Add PAM (ablation) | HaWoR · HOT3D | W-MPJPE: 36.32 -> 33.20 | Helped |
| Use the learned motion infiller instead of last-pose replication (ablation) | Hand motion completion · HOT3D validation, invisible sequences | FID: 1.52 -> 0.57 | Helped |
| Use the learned motion infiller instead of LERP (ablation) | Hand motion completion · HOT3D validation, invisible sequences | FID: 1.42 -> 0.57 | Helped |
Limitations
- Runtime is still far from real-time.
- Relies on an off-the-shelf hand detector and tracker and a multi-stage pipeline with DROID-SLAM and Metric3D.
- Evaluated on only two benchmarks (DexYCB, HOT3D).
Experimental setup
Datasets: HOT3D DexYCB
Benchmarks: DexYCB HOT3D
Metrics: PA-MPJPE AUC W-MPJPE WA-MPJPE RTE Accel FID ATE ATE-S
Baselines: WiLoR HaMeR HandDGP Deformer HMP-SLAM DROID-SLAM WiLoR-SLAM HaMeR-SLAM HandDGP-SLAM Spurr et al. MeshGraphormer SemiHandObj
Builds on: WiLoR DROID-SLAM Metric3D MANO SLAHMR
Closest papers by meaning
- StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video (NeurIPS 2026 · similarity 0.90)
- Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera (CVPR 2025 · similarity 0.90)
- Estimating Body and Hand Motion in an Ego-sensed World (CVPR 2025 · similarity 0.88)
- WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · similarity 0.87)
- The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.86)
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.