Research map / 3D vision / Human pose estimation
Depth-based hand pose regression
Estimates 3D hand pose from depth or RGB images using CNN regression, dense prediction and neural rendering.
84 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Human pose estimation, keyword cluster “object pose · 6d · hand”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 2 of 5 tracked benchmarks.
21 papers in 2023–24 and 19 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 7.7% to 6.3%.
Open questions the data points to
- FreiHAND (F@5): best went from 0.785 (up to 2024) to 0.825 in 2025 — WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations)
- FreiHAND (PA-MPVPE): best went from 5.5 (up to 2024) to 5.1 in 2025 — WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations)
- Papers report Hampali et al. on HO-3D (F@5) anywhere from 50.6 to 57.4: comparisons against it depend on whose number you take.
What looks settled
- FreiHAND (F@15): best up to 2024 0.99, best in 2025–26 0.993 from 2 papers, less than 1% better.
No longer reported by papers here: FreiHAND (F@5 mm, last in 2024); FreiHAND (F@15 mm, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs (CVPR 2016 · 272 citations)
- 2017 · 3D Convolutional Neural Networks for Efficient and Robust Hand Pose Estimation From Single Depth Images (CVPR 2017 · 311 citations)
- 2018 · GANerated Hands for Real-Time 3D Hand Tracking From Monocular RGB (CVPR 2018 · 555 citations)
- 2019 · H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactions (CVPR 2019 · 281 citations)
- 2020 · HOnnotate: A Method for 3D Annotation of Hand and Object Poses (CVPR 2020 · 427 citations)
- 2021 · DexYCB: A Benchmark for Capturing Hand Grasping of Objects (CVPR 2021 · 281 citations)
- 2022 · Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimation (CVPR 2022 · 162 citations)
- 2023 · AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimation (CVPR 2023 · 66 citations)
- 2024 · Reconstructing Hands in 3D with Transformers (CVPR 2024 · 151 citations)
- 2025 · WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations)
- 2026 · HandMCM: Multi-modal Point Cloud-based Correspondence State Space Model for 3D Hand Pose Estimation (AAAI 2026 · 1 citation)
Earliest papers here
- Robust 3D Hand Pose Estimation in Single Depth Images: From Single-View CNN to Multi-View CNNs (CVPR 2016 · 272 citations)
- DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed With Deep Features (CVPR 2016 · 166 citations)
- Efficiently Creating 3D Training Data for Fine Hand Pose Estimation (CVPR 2016 · 90 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
FreiHAND · F@5 (higher is better)
5 papers, 2022–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 0.784 | MobRecon: Mobile-Friendly Hand Mesh Reconstruction From Monocular Image (CVPR 2022 · 109 citations) |
| 2024 | 0.785 | Reconstructing Hands in 3D with Transformers (CVPR 2024 · 151 citations) |
| 2025 | 0.825 | WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations) |
FreiHAND · F@15 (higher is better)
5 papers, 2022–2025 · small gains only
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 0.986 | MobRecon: Mobile-Friendly Hand Mesh Reconstruction From Monocular Image (CVPR 2022 · 109 citations) |
| 2024 | 0.99 | Reconstructing Hands in 3D with Transformers (CVPR 2024 · 151 citations) |
| 2025 | 0.993 | WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations) |
FreiHAND · PA-MPVPE (lower is better)
4 papers, 2024–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2024 | 5.5 | Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mamba (NeurIPS 2024 · 18 citations) |
| 2025 | 5.1 | WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations) |
FreiHAND · F@5 mm (higher is better)
4 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 0.614 | Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild (CVPR 2020 · 220 citations) |
| 2023 | 0.654 | Handy: Towards a High Fidelity 3D Hand Shape and Appearance Model (CVPR 2023 · 19 citations) |
| 2024 | 0.783 | Cross-Constrained Progressive Inference for 3D Hand Pose Estimation with Dynamic Observer-Decision-Adjuster Networks (AAAI 2024 · 1 citation) |
FreiHAND · F@15 mm (higher is better)
4 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 0.966 | Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild (CVPR 2020 · 220 citations) |
| 2023 | 0.971 | Handy: Towards a High Fidelity 3D Hand Shape and Appearance Model (CVPR 2023 · 19 citations) |
| 2024 | 0.991 | Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mamba (NeurIPS 2024 · 18 citations) |
What has been tried
77 papers here report 858 measured changes (a component added, removed or swapped, compared with a baseline): helped 79%, no clear effect 4%, hurt 15%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase training examples by 4× | HInt New Days, all joints · PCK@0.05 | 16.9 -> 31.3 | Helped | Reconstructing Hands in 3D with Transformers (CVPR 2024 · 151 citations) |
| add the large-scale WHIM training data rather than train on Oxford-Hands | Oxford-Hands and WHIM · mAP | 40.34 -> 48.98; 35.29 -> 53.79 | Helped | WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild (CVPR 2025 · 49 citations) |
| Use two-view input instead of single view for hand tracking | HOT3D and UmeTrack hand tracking · MKPE (mm) | HOT3D 15.4 -> 10.9; UmeTrack 13.4 -> 9.5 (about 41% improvement in two-view mode) | Helped | HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · 35 citations) |
| use global 3D SDFs as the intermediate representation instead of 2D keypoints | DexYCB test · MJE | 14.9 -> 10.1 | Helped | HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields (CVPR 2024 · 20 citations) |
| add Token Sampler Branch to fusion | FreiHAND · F@5mm | 0.722 -> 0.738 | Helped | Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mamba (NeurIPS 2024 · 18 citations) |
| Add ABM and RGR pseudo-labeling modules | Dah → Dah, Dah-Headset1 · Mono-M (MPJPE, mm) | 24.96 -> 20.16 | Helped | Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimation (CVPR 2024 · 12 citations) |
| increase the fusion stages from 1 to 3 | DexYCB S0 · MPJPE (mm) | 7.23 -> 6.89 | Helped | Keypoint Fusion for RGB-D Based 3D Hand Pose Estimation (AAAI 2024 · 10 citations) |
| Add pretrained ViT | HOT3D · PA-MPJPE | 7.59 -> 4.79 | Helped | HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos (CVPR 2025 · 8 citations) |
| Use temporal smoothing or TOCH instead of BioPR | DexYCB · MPJPE | less than a 0.8mm improvement | Hurt | MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints (CVPR 2024 · 8 citations) |
| use instance-specific queries instead of unified queries | HInt New Days · PCK@0.1 | 80.7 -> 82.5 | Helped | HandOS: 3D Hand Reconstruction in One Stage (CVPR 2025 · 7 citations) |
| replace 3D hand-pose representation with hand angles | Evaluation · R² | 88.86 ± 11.92% -> 84.22 ± 17.11% | Hurt | Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation (NeurIPS 2024 · 6 citations) |
| remove coarse UV-pressure loss Lc | EgoPressure ego. & exo. · UV Press. Contact IoU | 33.12 -> 26.24 | Hurt | EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision (CVPR 2025 · 5 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Hampali et al. | HO-3D | F@5 | 50.6 – 57.4 | 2 papers, e.g. CVPR 2021 and CVPR 2025 |
What reviewers will expect
Datasets and baselines named in the 15 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- DexYCB · 8 of 15 papers
- FreiHAND · 5 of 15 papers
- HO3D · 3 of 15 papers
- ObMan · 2 of 15 papers
- HO3D-v2 · 2 of 15 papers
- MOW · 2 of 15 papers
- RHD · 2 of 15 papers
- HOT3D · 2 of 15 papers
Baselines
- MeshGraphormer · 4 of 15 papers
- HaMeR · 4 of 15 papers
- METRO · 3 of 15 papers
- SimpleHand · 3 of 15 papers
- HandOccNet · 3 of 15 papers
- MobRecon · 3 of 15 papers
- S2HAND · 2 of 15 papers
- AMVUR · 2 of 15 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 40 papers; from OpenAlex and the papers' affiliations).
- Meta · 4 papers
- Tencent · 2 papers
- The University of Tokyo · 2 papers
- National University of Singapore · 2 papers
- Department of Computer Science and Engineering · 2 papers
- Imperial College London · 2 papers
- Microsoft · 2 papers
- Shanghai Jiao Tong University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Learned camera pose estimation (96 papers)
- Category-level object pose estimation (92 papers)
- Instance-level 6D object pose (74 papers)
- Category-level pose estimation (54 papers)
Elsewhere in the topic
- Physics and sensor-based motion capture (125 papers)
- Monocular 2D-to-3D pose lifting (90 papers)
- Transformer-based multi-view 3D pose (83 papers)
- Human shape and pose regression (80 papers)
- Temporal video pose estimation (76 papers)
- Human mesh recovery from images (58 papers)
- Multi-person pose estimation (51 papers)
- Heatmap and coordinate pose encoding (50 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).