Research map / 3D vision / Human pose estimation
Dual-Diffusion for Binocular 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dual-Diffusion, a diffusion model that denoises 2D binocular poses and 3D triangulated poses jointly (via geometric projection/reprojection) to refine binocular 3D human pose estimates, with Z-embedding and baseline-width normalization.
Problem
Binocular 3D human pose estimation has higher 3D uncertainty than multiview setups, and existing diffusion refinement methods need an unknown initial 3D uncertainty distribution.
Approach
Model the diffusion over 2D binocular poses with a known Gaussian uncertainty, triangulate to noisy 3D poses, denoise in 3D with a GCN-Transformer denoiser, and reproject to 2D; add Z-embedding (root depth) and baseline-width normalization so one model handles varying depths and baselines.
Key results
- On MHAD with RSB-Pose152* 2D detector, Dual-Diffusion reduces MPJPE from 29.78 mm (Triangulation) to 27.76 mm.
- On H36M at 384 resolution, Dual-Diffusion achieves 28.67 mm MPJPE vs 30.07 mm for RSB-Pose152, with only 0.74M denoiser parameters.
- Dual-Diffusion outperforms random-noise diffusion and 2D-Diff baselines in MPJPE across diffusion steps; BaseL-norm gives zero-shot gains of 4.25 mm and 3.1 mm on 100 mm and 300 mm baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Dual-Diffusion refinement to RSB-Pose152 2D poses | Triangulation with RSB-Pose152 on MHAD · MHAD | MPJPE (mm): 29.78 -> 27.76 | Helped |
| Apply Dual-Diffusion refinement to RSB-Pose152 2D poses | Triangulation with RSB-Pose152 on H36M · H36M | MPJPE (mm): 30.54 -> 28.67 | Helped |
| Add Z-embedding to Dual-Diff (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 28.20 -> 27.91 | Helped |
| Add BaseL-norm to Dual-Diff with Z-embedding (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 27.91 -> 27.76 | Helped |
| Add BaseL-norm for zero-shot baseline-width adaptation (ablation) | Dual-Diff with ResNet50 2D poses · MHAD, 100mm baseline | MPJPE (mm): 103.87 -> 88.32 | Helped |
| Add BaseL-norm for zero-shot baseline-width adaptation (ablation) | Dual-Diff with ResNet50 2D poses · MHAD, 300mm baseline | MPJPE (mm): 62.13 -> 51.26 | Helped |
| Initialize diffusion from random noise instead of the 2D-derived uncertainty distribution (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 28.20 -> 328.82 | Hurt |
| Denoise 2D poses directly instead of denoising 3D poses (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 28.20 -> 29.40 | Hurt |
| Vary diffusion steps T from 25 to 50, 75, 100, or 125 (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 28.20; 28.31; 28.17; 28.19; 28.25 | No clear effect |
| Increase inference iterations from K=1 to K=2, 5, 10, or 20 (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 28.20 -> 28.36; 28.80; 28.97; 29.08 | Hurt |
| Use 3D supervision instead of 2D supervision (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 27.76 -> 29.38 | Hurt |
| Add 3D supervision to 2D supervision (ablation) | Dual-Diff on MHAD with RSB-Pose152* 2D poses · MHAD | MPJPE (mm): 27.76 -> 27.73 | Helped |
Surprising findings
- Denoising with 3D-domain priors and only K=1 reverse iteration performed best; more iterations degraded MPJPE.
- Dual-Diffusion improves a 3D-domain model's 2D joint detection rate (e.g. 2.32% on RSB-Pose50) despite targeting 3D.
Limitations
- 3D uncertainty region is estimated via simulation and boundary analysis, which the authors acknowledge may lack rigor.
- Denoiser operates on root-relative poses and does not optimize the root joint directly.
- Gains diminish with more views (multiview improvements smaller than binocular).
Experimental setup
Datasets: MHAD H36M
Benchmarks: MHAD H36M
Metrics: MPJPE BL Sym JDR
Baselines: Triangulation RSB-Pose TPPT Epipolar_Tri Algebraic_Tri AdaFuse DiffPose random-noise diffusion 2D-Diff
Builds on: DDPM DDIM DiffPose Triangulation
Closest papers by meaning
- $\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.91)
- Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoiser (AAAI 2024 · similarity 0.87)
- Object Pose Estimation via the Aggregation of Diffusion Features (CVPR 2024 · similarity 0.86)
- Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.86)
- DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans (CVPR 2024 · similarity 0.86)
- Lifting by Image – Leveraging Image Cues for Accurate 3D Human Pose Estimation (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.