Research map / 3D vision / Human pose estimation
Physics and sensor-based motion capture
Recovers human motion from video or inertial sensors using physical constraints and object interaction.
125 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Human pose estimation, keyword cluster “body · motion · human mesh”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
42 papers in 2023–24 and 52 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 15.5% to 17.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Rolling Rotations for Recognizing Human Actions From 3D Skeletal Data (CVPR 2016 · 224 citations)
- 2017 · Multiple People Tracking by Lifted Multicut and Person Re-Identification (CVPR 2017 · 581 citations)
- 2018 · Neural Kinematic Networks for Unsupervised Motion Retargetting (CVPR 2018 · 256 citations)
- 2019 · Monocular Total Capture: Posing Face, Body, and Hands in the Wild (CVPR 2019 · 350 citations)
- 2020 · DeepCap: Monocular Human Performance Capture Using Weak Supervision (CVPR 2020 · 219 citations)
- 2021 · Populating 3D Scenes by Learning Human-Scene Interaction (CVPR 2021 · 140 citations)
- 2022 · Physical Inertial Poser (PIP): Physics-Aware Real-Time Human Motion Tracking From Sparse Inertial Sensors (CVPR 2022 · 214 citations)
- 2023 · Generating Holistic 3D Human Motion From Speech (CVPR 2023 · 160 citations)
- 2024 · WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion (CVPR 2024 · 98 citations)
- 2025 · Estimating Body and Hand Motion in an Ego-sensed World (CVPR 2025 · 9 citations)
- 2026 · RacketVision: A Multiple Racket Sports Benchmark for Unified Ball and Racket Analysis (AAAI 2026 · 2 citations)
Earliest papers here
- Rolling Rotations for Recognizing Human Actions From 3D Skeletal Data (CVPR 2016 · 224 citations)
- Multiple People Tracking by Lifted Multicut and Person Re-Identification (CVPR 2017 · 581 citations)
- Seeing Invisible Poses: Estimating 3D Body Pose From Egocentric Video (CVPR 2017 · 107 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
AIST++ · MPJPE (lower is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 105.5 | Differentiable Dynamics for Articulated 3D Human Motion Reconstruction (CVPR 2022 · 37 citations) |
| 2023 | 33.3 | HybridCap: Inertia-Aid Monocular Capture of Challenging Human Motions (AAAI 2023 · 13 citations) |
What has been tried
83 papers here report 815 measured changes (a component added, removed or swapped, compared with a baseline): helped 81%, no clear effect 4%, hurt 13%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add feature integration to WHAM | EMDB 2 · MPJPE | 69.0 -> 59.3 | Helped | WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion (CVPR 2024 · 98 citations) |
| add hand representations relative to the head coordinate frame | AMASS protocol1, HMD · H-PE | 2.36 -> 1.65 | Helped | HMD-Poser: On-Device Real-time Human Motion Tracking from Scalable Sparse Observations (CVPR 2024 · 24 citations) |
| add energy-metric-based sampling during training | TotalCaptureReal · GA[°] | 15.4 -> 14.4 | Helped | DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion (CVPR 2024 · 23 citations) |
| Add floor-level updating to Fest | Walking downstairs · Full PE | 12.35 -> 6.53 | Helped | Mocap Everyone Everywhere: Lightweight Motion Capture With Smartwatches and a Head-Mounted Camera (CVPR 2024 · 21 citations) |
| Add noise-guided latent space learning | DSreal Loose, all participants · angular error (degree) | 21.54 ± 6.52 -> 19.83 ± 6.79 | Helped | Loose Inertial Poser: Motion Capture with IMU-attached Loose-Wear Jacket (CVPR 2024 · 20 citations) |
| Add EgoWholeBody training data | SceneEgo test dataset · MPJPE | 75.10 -> 64.19 | Helped | Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement (CVPR 2024 · 19 citations) |
| Train the Transformer encoder-decoder with reconstruction loss | Human3.6M test · ACCL | 15.4 -> 2.5 | Helped | PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos (CVPR 2024 · 17 citations) |
| Add a segmentation decoder | EE3D-R · MPJPE | 111.01 -> 108.85 | Helped | EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams (CVPR 2024 · 16 citations) |
| Add temporal foot consistency loss Lt | MMVP · MPJPE | 87.0 -> 83.0 | Helped | MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors (CVPR 2024 · 15 citations) |
| Add the Neural Descent module | Dynamic Human3.6M · W-MPJPE | 644.8 -> 605.4 | Helped | ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning (CVPR 2024 · 13 citations) |
| add vision input | Synthetic-Test · Succ | 73.0% -> 94.3% | Helped | Real-Time Simulated Avatar from Head-Mounted Sensors (CVPR 2024 · 11 citations) |
| Add interpenetration loss to mesh optimization | Hugging, Wrestling, Dancing, Karate, MMA · mP2S | reduced by 44.7 mm on average | Helped | Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions (NeurIPS 2024 · 11 citations) |
What reviewers will expect
Datasets and baselines named in the 21 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- AMASS · 8 of 21 papers
- RICH · 3 of 21 papers
- Human3.6M · 2 of 21 papers
- BEDLAM · 2 of 21 papers
- AIST · 2 of 21 papers
Baselines
- WHAM · 7 of 21 papers
- AvatarPoser · 3 of 21 papers
- SLAHMR · 3 of 21 papers
- GVHMR · 3 of 21 papers
- AvatarJLM · 2 of 21 papers
- LiveHPS · 2 of 21 papers
- TRACE · 2 of 21 papers
- AMP · 2 of 21 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 94 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 6 papers
- ShanghaiTech University · 5 papers
- Stanford University · 5 papers
- Max Planck Institute for Intelligent Systems · 4 papers
- Shanghai Jiao Tong University · 3 papers
- Meta · 3 papers
- University of Science and Technology Beijing · 2 papers
- ETH Zurich · 2 papers
- Harbin Institute of Technology · 2 papers
- Chinese Academy of Sciences · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Human shape and pose regression (80 papers)
- Human mesh recovery from images (58 papers)
- Human motion prediction (38 papers)
Elsewhere in the topic
- Learned camera pose estimation (96 papers)
- Category-level object pose estimation (92 papers)
- Monocular 2D-to-3D pose lifting (90 papers)
- Depth-based hand pose regression (84 papers)
- Transformer-based multi-view 3D pose (83 papers)
- Temporal video pose estimation (76 papers)
- Instance-level 6D object pose (74 papers)
- Category-level pose estimation (54 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).