Research map / 3D vision / Human pose estimation
FinePOSE: Fine-Grained Prompt-Driven 3D Human Pose Estimation via Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FinePOSE, a diffusion-based 3D human pose estimation denoiser whose learnable fine-grained body-part prompts (via frozen CLIP text encoder) guide denoising, evaluated on Human3.6M, MPI-INF-3DHP and EgoHumans.
Problem
Monocular 3D human pose estimation from 2D keypoints suffers from depth ambiguity, pose complexity and poor generalization, and prior methods ignore textual/kinematic knowledge and fine-grained body-part guidance.
Approach
A fine-grained part-aware prompt learning block couples CLIP text prompts (action class, speed, person, head, body, arms, legs) with learnable modifier vectors; these prompts are injected into a MixSTE-based diffusion denoiser via cross-attention (FPC) and timestamp-conditioned feature modulation (PTS).
Key results
- Human3.6M with CPN detector: 31.9mm MPJPE and 25.0mm P-MPJPE, beating D3DP (35.4mm) by 3.5mm.
- MPI-INF-3DHP with GT 2D keypoints: 26.2mm MPJPE, 98.9 PCK, 80.0 AUC, improving over GLA-GCN by 1.6mm MPJPE.
- EgoHumans multi-human: 34.3mm average MPJPE vs. 35.4mm for D3DP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use manually designed action-class prompts instead of no prompts (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 37.2 -> 35.8 | Helped |
| Add a learnable prompt combined with the action class (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 37.2 -> 36.2 | Helped |
| Use action class and coarse-grained information in the prompt (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 36.2 -> 34.7 | Helped |
| Use learnable prompts without manual design (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 36.2 -> 34.6 | Helped |
| Use fine-grained part-aware prompts rather than coarse prompt variants (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 31.9 vs 34.6 | Helped |
| Add FPP to the baseline (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 37.2 -> 35.3 | Helped |
| Add only PTS without FPP and FPC (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 37.2 -> 37.1 | No clear effect |
| Remove FPC (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 31.9 -> 35.7 | Hurt |
| Remove PTS (ablation) | FinePOSE · Human3.6M (DET) | MPJPE: 31.9 -> 36.6 | Hurt |
| Use FinePOSE rather than D3DP | 3D HPE · Human3.6M (DET) | MPJPE: 35.4 -> 31.9 | Helped |
| Use FinePOSE rather than GLA-GCN | 3D HPE · MPI-INF-3DHP test | MPJPE: 27.8 -> 26.2 | Helped |
| Extend FinePOSE to multi-human pose estimation rather than use D3DP | 3D multi-human pose estimation · EgoHumans average | MPJPE: 35.4 -> 34.3 | Helped |
Surprising findings
- Manually designed prompts (M-Prompt) outperform learnable-only prompts when combined with textual action class (S-Prompt degrades relative to M-Prompt).
Limitations
- Not explicitly designed for multi-person scenarios.
- Diffusion-based 3D HPE is relatively computationally expensive.
- Multi-human extension is a simple post-integration of single-person predictions.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP EgoHumans
Benchmarks: Human3.6M MPI-INF-3DHP EgoHumans
Metrics: MPJPE P-MPJPE PCK AUC
Baselines: TCN Anatomy P-STMO MixSTE PoseFormerV2 MHFormer Diffpose GLA-GCN ActionPrompt MotionBERT D3DP
Builds on: MixSTE D3DP CLIP CoOp MotionDiffuse
Closest papers by meaning
- $\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.87)
- Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoiser (AAAI 2024 · similarity 0.86)
- Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation (NeurIPS 2024 · similarity 0.86)
- FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidance (CVPR 2025 · similarity 0.86)
- ChatPose: Chatting about 3D Human Pose (CVPR 2024 · similarity 0.86)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.