Research map / 3D vision / Human pose estimation
HardMo: A Large-Scale Hardcase Dataset for Motion Capture
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds HardMo, a 7M-image dance and martial-arts dataset with an automatic crawl-annotate-hardcase-mining pipeline, and retrains HMR and 4DHumans on it, showing gains on hand/foot hardcases.
Problem
Monocular human mesh recovery methods fail on unusual poses in dance and martial arts because existing training datasets lack such hand and foot postures and carry incorrect SMPL annotations for them.
Approach
An automatic pipeline that crawls online dance/martial-arts videos, pseudo-labels them with RTMPose and 4DHumans, mines hand and foot hardcases using joint-angle criteria, and refines their SMPL labels with a learned optimization prior, yielding the HardMo dataset and its HardMo-Hand and HardMo-Foot subsets.
Key results
- HardMo-HMR reaches 36.0 mm MPJPE on the HardMo benchmark, versus 36.6 mm for 4DHumans trained on InstaVariety and 61.1 mm for HMR trained on InstaVariety.
- On HardMo-Foot P1, HardMo-HMR gets 34.9 mm foot MPJPE, about 7x better than ProHMR (213.6 mm) and about 3x better than 4DHumans (88.1 mm).
- On HardMo-Hand, HardMo-HMR reaches PCK@0.01-hand of 0.36 versus 0.11 for 4DHumans and 0.03 for ProHMR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train HMR on HardMo instead of using HMR trained on InstaVariety | HMR · HardMo | MPJPE: 61.1 -> 36.0 | Helped |
| Train HMR on HardMo instead of using 4DHumansb | HMR · HardMo | MPJPE: 36.6 -> 36.0 | Helped |
| Train 4DHumans on HardMo instead of using 4DHumansb | 4DHumans · HardMo | MPJPE: 36.6 -> 26.0 | Helped |
| Add optimized labels during HMR training (ablation) | HardMo-HMR · HardMo-Foot P1 | MPJPE-foot: 58.3 -> 34.9 | Helped |
| Add optimized labels during HMR training (ablation) | HardMo-HMR · HardMo-Foot P2 | MPJPE-foot: 58.1 -> 39.5 | Helped |
| Train HMR on HardMo-Foot | HMR · HardMo-Foot P1 | MPJPE-foot: 213.6 -> 34.9 | Helped |
| Train 4DHumans on HardMo-Hand | 4DHumans · HardMo-Hand | PCK@0.01-hand: 0.11 -> 0.54 | Helped |
| Train HMR on HardMo-Hand | HMR · HardMo-Hand | PCK@0.01-hand: 0.03 -> 0.36 | Helped |
Surprising findings
- HMR, an early method, retrained on HardMo outperforms the current state-of-the-art 4DHumans on the HardMo benchmarks.
- Training on the mixture with common datasets (HardMo-HMR†) was worse than HardMo alone for the HMR variant with 46.0 mm vs 36.0 mm MPJPE, suggesting common datasets are inadequate for this domain.
Limitations
- Focuses only on foot and hand misalignment; other challenges such as self-occlusion are not addressed.
- Hardcase labels rely on pseudo-labels from off-the-shelf regressors (4DHumans) and keypoint detectors, with optimization-based refinement.
Experimental setup
Datasets: HardMo HardMo-Foot HardMo-Hand InstaVariety AVA AI Challenger
Benchmarks: HardMo HardMo-Foot HardMo-Hand
Metrics: MPJPE PA-MPJPE PCK@0.01 PCK@0.05
Baselines: HMR ProHMR CLIFF 4DHumans
Builds on: HMR 4DHumans EFT SMPLify RTMPose
Closest papers by meaning
- GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities (CVPR 2025 · similarity 0.85)
- M3GYM: A Large-Scale Multimodal Multi-view Multi-person Pose Dataset for Fitness Activity Understanding in Real-world Settings (CVPR 2025 · similarity 0.84)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.83)
- DanceFix: An Exploration in Group Dance Neatness Assessment Through Fixing Abnormal Challenges of Human Pose (AAAI 2025 · similarity 0.83)
- Lifting Motion to the 3D World via 2D Diffusion (CVPR 2025 · similarity 0.83)
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.