Research map / Point clouds / Point cloud classification
Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DAPT, a point-transformer pose estimator with learnable joint anchors, multi-density exchange, and 1D heatmaps, pre-trained on SMPL-based LiDAR ray-cast synthetic data with random placement and laser-level masking for single-frame LiDAR 3D human pose estimation.
Problem
Robust 3D human pose estimation from noisy, sparse, and occluded single-frame LiDAR point clouds without temporal info, multi-modal fusion, or SMPL optimization.
Approach
Learnable joint anchors that progressively exchange information with multi-scale point features (MDE module) to get density-robust joint features, decoded as 1D heatmaps per axis; plus a synthetic LiDAR pre-training pipeline with randomized human placement, ground mesh, and laser-level masking.
Key results
- On Waymo, MPJPE 51.59 mm vs. LPFormer 61.60 mm (10.0 mm reduction).
- On SLOPER4D, MPJPE 28.01 mm vs. PRN 48.76 mm (20.7 mm reduction).
- On LidarHuman26M, MPJPE 73.47 mm, close to NE's 72.23 mm, with PA-MPJPE 58.40 mm vs. NE 61.67 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DAPT instead of LPFormer | LiDAR-based pose estimation · Waymo | MPJPE: 10.0mm | Helped |
| Use DAPT instead of PRN | LiDAR-based pose estimation · SLOPER4D | MPJPE: 20.7mm | Helped |
| Use DAPT instead of PRN | LiDAR-based pose estimation · HumanM3 | MPJPE: 10.8mm | Helped |
| Use DAPT instead of NE | LiDAR-based pose estimation · LiDARHuman26M | MPJPE | Hurt |
| Add joint anchors and a 4-layer Transformer (ablation) | PRN baseline on Waymo · Waymo | MPJPE: 68.5 -> 54.3 | Helped |
| Add MDE to the anchor model (ablation) | PRN baseline on Waymo · Waymo | MPJPE: 54.3 -> 52.3 | Helped |
| Replace coordinate decoding with heatmap decoding (ablation) | Anchor model on Waymo · Waymo | MPJPE: 0.6mm | Helped |
| Enable anchors, MDE, and heatmap decoding (ablation) | PRN baseline on Waymo · Waymo | MPJPE: 68.5 -> 51.7 | Helped |
| Add pre-training (ablation) | DAPT on Waymo · Waymo | MPJPE: 59.2 -> 52.9 | Helped |
| Add scene resampling (ablation) | Pre-trained DAPT on Waymo · Waymo | MPJPE: 52.9 -> 52.4 | Helped |
| Add laser-level masking (ablation) | Scene-resampled DAPT on Waymo · Waymo | MPJPE: 52.4 -> 51.7 | Helped |
| Vary r_keep from 0.5 to 0.9 (ablation) | Pre-trained DAPT on Waymo · Waymo | MPJPE: 53.0; 51.7; 51.9; 52.5; 52.4 | Mixed |
Surprising findings
- Pre-training with the proposed synthesis pipeline on LidarHuman26M-based data gave a slightly worse Waymo-level MPJPE than the Waymo-specific variant, and on LiDARHuman26M the FreeMotion-synthesis pre-training was best (73.2 vs 73.4), indicating the gains are not uniform across datasets.
Limitations
- Left-right ambiguity of the body remains unresolved
- Multi-frame jitter is not addressed since only single-frame input is used
- Gains on LiDARHuman26M MPJPE are not superior to NE
Experimental setup
Datasets: LidarHuman26M SLOPER4D HumanM3 Waymo Open Dataset v2 SMPL real-world captured database (synthetic pre-training)
Benchmarks: Waymo Open Dataset v2 HumanM3 LidarHuman26M SLOPER4D
Metrics: MPJPE PA-MPJPE PCK-3 PCK-5
Baselines: LidarCap NE LPFormer PRN LIPD FreeMotion NoiseMotion
Builds on: Point Transformer V3 PRN LiDARCap synthetic pre-training
Closest papers by meaning
- LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment (CVPR 2024 · similarity 0.90)
- Neighborhood-Enhanced 3D Human Pose Estimation with Monocular LiDAR in Long-Range Outdoor Scenes (AAAI 2024 · similarity 0.90)
- FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions (CVPR 2024 · similarity 0.86)
- Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation (NeurIPS 2024 · similarity 0.86)
- TULIP: Transformer for Upsampling of LiDAR Point Clouds (CVPR 2024 · similarity 0.86)
- LiDAR-Net: A Real-scanned 3D Point Cloud Dataset for Indoor Scenes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.