Research map / 3D vision / Human pose estimation
ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Collects AscendMotion, a multi-modal (RGB, LiDAR, IMU) rock-climbing motion dataset, and proposes ClimbingCap, a global motion recovery method that fuses RGB and LiDAR with separate coordinate decoders, post-processing optimization, and teacher-student semi-supervised training.
Problem
Global 3D human motion recovery for rock climbing, an off-ground motion, is hard because of complex poses and scarce large-scale labeled climbing data with world-coordinate trajectories.
Approach
Use RGB to estimate camera-space poses and LiDAR point clouds to estimate global translation, then post-process in world coordinates with climbing-specific losses (limb-weighted distance, scene-direction smoothing, visible-limb repair) and refine with teacher-student semi-supervised training on unlabeled climbing video.
Key results
- On AscendMotion vertical scenes, ClimbingCap reaches W-MPJPE 106.95 vs 1442.50 for GVHMR (best baseline in that metric was GVHMR/ImmFusion range; ClimbingCap significantly better than second-best).
- On CIMI4D without retraining, ClimbingCap achieves MPJPE 84.03 and PA-MPJPE 60.69 vs 152.56 and 91.13 for GVHMR.
- Ablation on AscendMotion: removing the scene-direction smoothing loss raises W-MPJPE from 78.99 to 120.11 (w/o LSDS 164.11), and removing semi-supervised training slightly worsens MPJPE from 75.45 to 77.43.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ClimbingCap instead of GVHMR | Global motion recovery · AscendMotion vertical scene | MPJPE: 124.60 -> 88.92 | Helped |
| Use ClimbingCap instead of GVHMR | Global motion recovery · AscendMotion vertical scene | WA-MPJPE: 1002.11 -> 85.26 | Helped |
| Use ClimbingCap instead of GVHMR | Global motion recovery · AscendMotion vertical scene | W-MPJPE: 1442.50 -> 106.95 | Helped |
| Use ClimbingCap instead of GVHMR | Global motion recovery · AscendMotion vertical scene | RTE: 7.91 -> 3.12 | Helped |
| Use ClimbingCap instead of GVHMR | Global motion recovery · CIMI4D | MPJPE: 152.56 -> 84.03 | Helped |
| Use ClimbingCap instead of LEIR | Global motion recovery · CIMI4D | WA-MPJPE: 228.1 -> 228.28 | Hurt |
| Use ClimbingCap instead of LEIR | Global motion recovery · CIMI4D | W-MPJPE: 370.74 -> 261.06 | Helped |
| Use ClimbingCap instead of LEIR | Global motion recovery · CIMI4D | RTE: 10.6 -> 8.17 | Helped |
| Add LiDAR input alongside RGB (ablation) | ClimbingCap · AscendMotion | MPJPE: 105.67 -> 75.45 | Helped |
| Add limb weight differentiation loss (ablation) | ClimbingCap · AscendMotion | MPJPE: 80.46 -> 75.45 | Helped |
| Add speed-direction smoothing loss (ablation) | ClimbingCap · AscendMotion | MPJPE: 99.13 -> 75.45 | Helped |
| Add visible limb repair loss (ablation) | ClimbingCap · AscendMotion | MPJPE: 91.10 -> 75.45 | Helped |
Surprising findings
- Global RGB methods WHAM and GVHMR perform well in camera-coordinate metrics but poorly in world-coordinate metrics on climbing.
- GVHMR performs poorly on vertical climbing because it predicts motion direction from horizontal velocity.
Limitations
- Semi-supervised module gives only a modest gain according to the ablation.
- Dataset annotation relies on IMU initialization and manual repair by four observers, which is labor-intensive.
Experimental setup
Datasets: AscendMotion CIMI4D
Benchmarks: AscendMotion CIMI4D
Metrics: MPJPE PA-MPJPE PVE PCK0.3 ACCEL WA-MPJPE W-MPJPE RTE Jitter T-Error
Baselines: TRACE SLAHMR WHAM GVHMR LiDARCapV2 LiveHPS ImmFusion FusionPose LEIR
Builds on: WHAM GVHMR SLAHMR LiDARCapV2 IPNet
Closest papers by meaning
- WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion (CVPR 2024 · similarity 0.85)
- Lifting Motion to the 3D World via 2D Diffusion (CVPR 2025 · similarity 0.85)
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.84)
- RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method (CVPR 2024 · similarity 0.83)
- Lifting by Image – Leveraging Image Cues for Accurate 3D Human Pose Estimation (AAAI 2024 · similarity 0.82)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.