Research map / 3D vision / Human pose estimation
DiffusionRegPose: Enhancing Multi-Person Pose Estimation using a Diffusion-Based End-to-End Regression Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Converts a one-stage end-to-end keypoint regression pose estimator (ED-Pose-style decoder) into a DDIM-style diffusion sampling process over noisy pose candidates, with probabilistic completion of invisible keypoints, evaluated on COCO and CrowdPose.
Problem
One-stage deterministic keypoint regression methods for multi-person pose estimation miss or falsely detect people in crowded or occluded scenes because they cannot model pose ambiguity.
Approach
Treat pose estimation as denoising a set of noisy pose candidates conditioned on image tokens (inspired by DiffusionDet), with a Gaussian-likelihood (MLE via least squares) completion of invisible keypoints used to initialize the diffusion target and interleaved human-detection and keypoint attention.
Key results
- On COCO val2017 with ResNet-50, DiffusionRegPose reaches 72.5 AP, versus 71.6 for ED-Pose and 72.0 for GroupPose.
- On CrowdPose test, DiffusionRegPose reaches 72.7 AP and 64.9 APH, a 4.0 APH gain over ED-Pose (60.9).
- Ablation on CrowdPose shows a signal scale of 5.0 (default) gives 72.7 AP, and 100 instance queries outperform 50 (71.4) and 200 (72.1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use diffusion-based pose denoising and interaction learning instead of deterministic regression | one-stage pose estimation with ResNet-50 · COCO val2017 | AP: 71.6 -> 72.5 | Helped |
| use diffusion-based pose denoising and interaction learning instead of deterministic regression | one-stage pose estimation with ResNet-50 · CrowdPose test | APH: +4.0% | Helped |
| set signal scale to 5.0 instead of 1.0 (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.4 -> 72.7 | Helped |
| set signal scale to 10.0 instead of 1.0 (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.4 -> 72.7 | Helped |
| use signal scale 0.1 instead of 1.0 (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.4 -> 72.3 | Hurt |
| use empty pose padding instead of noise padding (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.6 -> 72.7 | Helped |
| use empty pose padding instead of mean pose padding (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.5 -> 72.7 | Helped |
| use GT repeat padding instead of empty pose padding (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.7 -> 72.7 | No clear effect |
| increase instance queries from 50 to 100 (ablation) | DiffusionRegPose · CrowdPose test | AP: 71.4 -> 72.7 | Helped |
| increase instance queries from 100 to 200 (ablation) | DiffusionRegPose · CrowdPose test | AP: 72.7 -> 72.1 | Hurt |
| complete invisible keypoints to rational coordinates instead of leaving their coordinates at zero (ablation) | DiffusionRegPose training · CrowdPose | AP | Helped |
Surprising findings
- The optimal signal scale (5.0 or 10.0) is much larger than the values typically used for image generation (1.0) and segmentation (0.1).
- Simple empty-pose or GT-replication padding performed as well as or better than noise or mean-pose padding.
Limitations
- Human detection AP (46.5 on COCO) is well below top-down detectors (56.4), and the pose gap to the best top-down method remains 3.4 AP on COCO.
- Gains on COCO are modest (0.9 AP over ED-Pose), and the overall approach is built on ED-Pose's decoder.
- Results and ablations are reported for ResNet-50 only; the paper states code will be available but does not give a release statement beyond the abstract.
Experimental setup
Datasets: MS COCO train2017 CrowdPose trainval
Benchmarks: MS COCO val2017 CrowdPose test
Metrics: AP AP50 AP75 APM APL APb APE APH
Baselines: ED-Pose GroupPose QueryPose PETR DiffusionPose SimpleBaseline HRFormer-B SWAHR
Builds on: DiffusionDet ED-Pose DDIM DDPM
Closest papers by meaning
- DiffPose: Toward More Reliable 3D Pose Estimation (CVPR 2023 · similarity 0.89)
- Object Pose Estimation via the Aggregation of Diffusion Features (CVPR 2024 · similarity 0.88)
- Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models (ICLR 2024 · similarity 0.87)
- $\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.87)
- Cross-view Masked Diffusion Transformers for Person Image Synthesis (ICML 2024 · similarity 0.86)
- Confronting Ambiguity in 6D Object Pose Estimation via Score-Based Diffusion on SE(3) (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.