Research map / 3D vision / Human pose estimation
Monocular 3D Hand Mesh Recovery via Dual Noise Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes dual noise estimation (DNE), a probabilistic refinement of MANO-based hand meshes that models 2D and 3D vertex noise as Gaussians and refines camera parameters in closed form, evaluated on InterHand2.6M.
Problem
Monocular 3D hand mesh recovery from single images: parametric hand models have fixed topology and deterministic refinements struggle to align meshes with the image.
Approach
Treat mesh vertices and their image-plane projections as noisy and jointly refine them via learned Gaussian noise distributions sampled with the reparameterization trick, using image-aligned features and multi-view voxel max-pooled features for 3D noise, and refine the orthographic camera via ridge regression.
Key results
- On InterHand2.6M, Ours-GraphAttn reaches 8.40 mm MPJPE overall versus 9.63 mm for the prior best, and Ours-MLP reaches 8.78 mm.
- The full DNE module reduces the fitting baseline's MPVPE from 12.53 to 10.87 mm overall (M=3 DNE modules), and the baseline improves by more than 10% per the abstract.
- Using more DNE modules (M=3 vs M=1) steadily improves MPVPE-All from 11.67 to 10.87 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply dual noise estimation instead of the parametric fitting baseline (ablation) | MANO baseline · Interhand2.6M All (10% training data) | MPVPE: 12.53 -> 10.87 | Helped |
| Add 3D noise estimation (ablation) | MANO baseline · Interhand2.6M All (10% training data) | MPVPE: 12.53 -> 11.02 | Helped |
| Add 2D noise estimation (ablation) | MANO baseline · Interhand2.6M All (10% training data) | 2D MPVPE: 13.31 -> 12.45 | Helped |
| Add 3D noise estimation (ablation) | MANO baseline · Interhand2.6M All (10% training data) | 2D MPVPE: 13.31 -> 10.12 | Helped |
| Combine 2D and 3D noise estimation (ablation) | MANO baseline · Interhand2.6M All (10% training data) | 2D MPVPE: 13.31 -> 9.44 | Helped |
| Increase the number of DNE modules from one to three (ablation) | DNE hand mesh refinement · Interhand2.6M All (10% training data) | MPVPE: 11.67 -> 10.87 | Helped |
| Increase the number of DNE modules from one to three (ablation) | DNE hand mesh refinement · Interhand2.6M All (10% training data) | MPJPE: 11.33 -> 10.64 | Helped |
| Use Ours-MLP instead of the best prior method | Interhand2.6M hand mesh recovery · Interhand2.6M All | MPJPE (mm): 9.63 -> 8.78 | Helped |
| Use Ours-GraphAttn instead of the best prior method | Interhand2.6M hand mesh recovery · Interhand2.6M All | MPJPE (mm): 9.63 -> 8.40 | Helped |
Limitations
- Designed for generic inputs without special handling of single-hand versus interacting-hand images; cross-hand noise model left for future work.
- Ablations use only 10% of training data due to computational expense.
Experimental setup
Datasets: InterHand2.6M
Benchmarks: InterHand2.6M
Metrics: MPJPE MPVPE
Baselines: IntagHand Ours-GraphAttn (variants compared against prior SOTA) MANO fitting baseline
Builds on: MANO ResNet IntagHand
Closest papers by meaning
- HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions (CVPR 2024 · similarity 0.89)
- Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering (AAAI 2024 · similarity 0.88)
- SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit Meshes (NeurIPS 2025 · similarity 0.88)
- A Simple Baseline for Efficient Hand Mesh Reconstruction (CVPR 2024 · similarity 0.87)
- Learning Explicit Contact for Implicit Reconstruction of Hand-Held Objects from Monocular Images (AAAI 2024 · similarity 0.87)
- Reconstructing Hands in 3D with Transformers (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.