Research map / 3D vision / Human pose estimation
RAPTR: Radar-based 3D Pose Estimation using Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAPTR, a two-stage transformer decoder with pseudo-3D deformable attention that estimates radar-based 3D human poses from multi-view heatmaps using only 3D bounding-box and 2D keypoint labels.
Problem
Radar-based indoor 3D human pose estimation normally needs costly fine-grained 3D keypoint labels; the paper targets weak supervision using cheaper 3D BBox and 2D keypoint labels, with depth ambiguity as a key difficulty.
Approach
A two-stage (pose and joint) query decoder with pseudo-3D deformable attention, where 3D reference points and offsets are sampled and projected onto two radar views; a 3D template loss from BBox centroids initializes poses, then a 3D gravity loss and a 2D keypoint loss refine them.
Key results
- On HIBER WALK, RAPTR reaches 22.32 cm overall MPJPE, a 34.3% reduction versus HRRadarPose (33.96 cm).
- On HIBER MULTI, RAPTR reaches 18.99 cm MPJPE versus 33.19 cm for HRRadarPose (42.7% reduction).
- On MMVR P1S1, RAPTR reduces center distance to 31.41 cm from 136.14 cm for Person-in-WiFi 3D (76.9% reduction).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RAPTR instead of HRRadarPose | HIBER WALK · HIBER WALK | MPJPE (cm): 33.96 -> 22.32 | Helped |
| Use RAPTR instead of HRRadarPose | HIBER MULTI · HIBER MULTI | MPJPE (cm): 33.19 -> 18.99 | Helped |
| Use RAPTR instead of Person-in-WiFi 3D | MMVR P1S1 · MMVR P1S1 | Center distance (cm): 136.14 -> 31.41 | Helped |
| Add 3D constraints to K2D-only supervision at the joint decoder (ablation) | RAPTR · HIBER WALK and MULTI | MPJPE (cm): 381.18 $\pm$ 0.28 -> 22.32 $\pm$ 0.06 (WALK); 375.73 $\pm$ 6.31 -> 18.99 $\pm$ 0.16 (MULTI) | Helped |
| Add T3D loss at the pose decoder (ablation) | RAPTR · HIBER WALK | MPJPE (cm): 28.54 $\pm$ 4.57 -> 22.32 $\pm$ 0.06 | Helped |
| Add T3D loss at the pose decoder (ablation) | RAPTR · HIBER MULTI | MPJPE (cm): 57.90 $\pm$ 9.81 -> 18.99 $\pm$ 0.16 | Helped |
| Use T3D instead of K2D+G3D at the pose decoder (ablation) | RAPTR · HIBER WALK | MPJPE (cm): 27.49 $\pm$ 3.40 -> 22.32 $\pm$ 0.06 | Helped |
| Use T3D instead of K2D+G3D at the pose decoder (ablation) | RAPTR · HIBER MULTI | MPJPE (cm): 23.43 $\pm$ 3.44 -> 18.99 $\pm$ 0.16 | Helped |
| Add G3D loss at the joint decoder (ablation) | RAPTR · HIBER WALK | MPJPE (cm): 25.96 $\pm$ 4.95 -> 22.32 $\pm$ 0.06 | Helped |
| Add G3D loss at the joint decoder (ablation) | RAPTR · HIBER MULTI | MPJPE (cm): 25.83 $\pm$ 3.87 -> 18.99 $\pm$ 0.16 | Helped |
| Use pseudo-3D instead of decoupled 2D deformable attention (ablation) | RAPTR · HIBER WALK | MPJPE (cm): 23.25 $\pm$ 1.38 -> 22.32 $\pm$ 0.06 | Helped |
| Use pseudo-3D instead of decoupled 2D deformable attention (ablation) | RAPTR · HIBER MULTI | MPJPE (cm): 19.47 $\pm$ 0.95 -> 18.99 $\pm$ 0.16 | Helped |
Surprising findings
- Removing the 3D template loss and using only 2D keypoint and gravity loss at the pose decoder degrades MULTI error from 18.99 to 57.90 cm.
- Pseudo-3D deformable attention gives only marginal gains over decoupled 2D deformable attention (about 4% on WALK and 2.5% on MULTI).
Limitations
- Accuracy depends on the precision of 2D keypoint labels; 2D labels cannot disambiguate facing direction, so joints may bend in the wrong depth direction.
- Occlusion and human-to-human interference degrade performance in crowded scenes.
- MMVR lacks 3D keypoint labels, so evaluation uses approximate BBox-based metrics.
Experimental setup
Datasets: HIBER MMVR
Benchmarks: HIBER WALK HIBER MULTI MMVR P1S1
Metrics: MPJPE center distance edge length error
Baselines: Person-in-WiFi 3D HRRadarPose QRFPose Pose Lifting
Builds on: PETR Deformable DETR QRFPose ResNet
Closest papers by meaning
- RETR: Multi-View Radar Detection Transformer for Indoor Perception (NeurIPS 2024 · similarity 0.85)
- RCTrans: Radar-Camera Transformer via Radar Densifier and Sequential Decoder for 3D Object Detection (AAAI 2025 · similarity 0.83)
- PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting (NeurIPS 2025 · similarity 0.82)
- MVDoppler-Pose: Multi-Modal Multi-View mmWave Sensing for Long-Distance Self-Occluded Human Walking Pose Estimation (CVPR 2025 · similarity 0.82)
- Multiple View Geometry Transformers for 3D Human Pose Estimation (CVPR 2024 · similarity 0.82)
- Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.