Research map / 3D vision / Human pose estimation
Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2DHand, an unsupervised adaptation of a pre-trained single-view 3D hand pose estimator to arbitrary unlabeled dual-view camera pairs, using cross-view consensus (attention-based merging) and rotation-invariance (rotation-guided refinement) pseudo-labels.
Problem
Single-view 3D hand pose estimators suffer from limited field-of-view and depth ambiguity, while existing multi-view methods require costly multi-view labels and fixed, known camera parameters at test time.
Approach
Adapt a single-view estimator to dual views without labels or camera parameters by generating pseudo-labels from two stereo constraints: joint-wise attention-weighted cross-view consensus and temporal invariance of the inter-camera rotation, combined with a momentum teacher.
Key results
- In-dataset adaptation lowers overall Mono-M MPJPE from 24.96mm to 20.16mm (Headset1) and Dual-M from 22.67mm to 19.98mm.
- Under cross-dataset (synthetic-to-real) settings, S2DHand reaches 48.44mm Mono-M vs 56.18mm source-only on Headset1, outperforming SFDAHPE (54.06mm) and other unsupervised adaptation baselines.
- Ablation shows ABM alone and RGR alone each improve over baseline, and combining both gives the best results (20.16mm Mono-M on Headset1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dah → Dah, Overall, Dah-Headset1 | Mono-M (MPJPE, mm): 24.96 -> 20.16 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dah → Dah, Overall, Dah-Headset1 | Dual-M (MPJPE, mm): 22.67 -> 19.98 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dah → Dah, Overall, Dah-Headset2 | Mono-M (MPJPE, mm): 25.72 -> 21.67 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dah → Dah, Overall, Dah-Headset2 | Dual-M (MPJPE, mm): 23.75 -> 21.25 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dsyn → Dah, Overall, Dah-Headset1 | Mono-M (MPJPE, mm): 56.18 -> 48.44 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dsyn → Dah, Overall, Dah-Headset1 | Dual-M (MPJPE, mm): 50.59 -> 45.92 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dsyn → Dah, Overall, Dah-Headset2 | Mono-M (MPJPE, mm): 58.74 -> 53.11 | Helped |
| Adapt the single-view estimator to the dual-view camera pair | DetNet · Dsyn → Dah, Overall, Dah-Headset2 | Dual-M (MPJPE, mm): 54.72 -> 50.37 | Helped |
| Add ABM and RGR pseudo-labeling modules (ablation) | S2DHand · Dah → Dah, Dah-Headset1 | Mono-M (MPJPE, mm): 24.96 -> 20.16 | Helped |
| Add ABM and RGR pseudo-labeling modules (ablation) | S2DHand · Dah → Dah, Dah-Headset2 | Dual-M (MPJPE, mm): 23.75 -> 21.25 | Helped |
| Add RGR to ABM (ablation) | S2DHand · Dah → Dah, Dah-Headset1 | Mono-M (MPJPE, mm): 20.81 -> 20.16 | Helped |
| Add ABM to RGR (ablation) | S2DHand · Dah → Dah, Dah-Headset1 | Mono-M (MPJPE, mm): 21.89 -> 20.16 | Helped |
Surprising findings
- Source-free S2DHand outperforms the fine-tuning upper bound that uses target labels on Headset1 Dual-M (45.92mm vs 38.11mm for fine-tuning is actually worse; fine-tune Dual-M is 38.11mm, so fine-tuning is better in Dual-M), so this is not claimed; instead the paper notes gains are small when initial
Limitations
- Gains are small when the initial single-view estimator is inaccurate (e.g., cam 1,2 cross-dataset).
- Requires a rotation matrix initialization assumed accurate from the pre-trained estimator.
- Evaluated only on AssemblyHands with DetNet backbone and one dataset pair family.
Experimental setup
Datasets: AssemblyHands GANerated Hands Rendered Handpose
Benchmarks: AssemblyHands
Metrics: MPJPE (Mono-M) dual-view MPJPE (Dual-M)
Baselines: SFDAHPE RegDA DAGEN ADDA Fine-tune Source Only
Builds on: DetNet mean teacher Procrustes rotation estimation
Closest papers by meaning
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · similarity 0.87)
- StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video (NeurIPS 2026 · similarity 0.87)
- SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit Meshes (NeurIPS 2025 · similarity 0.86)
- Keypoint Fusion for RGB-D Based 3D Hand Pose Estimation (AAAI 2024 · similarity 0.86)
- HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos (CVPR 2025 · similarity 0.86)
- UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.