Research map / 3D vision / Human pose estimation
Sequential Joint Dependency Aware Human Pose Estimation with State Space Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a 2D-to-3D human pose estimator that adds a state space model (Mamba-style) module over sequentially serialized joints, with joint-dependent SSM parameters and pose-aware hidden state initialization.
Problem
Monocular 2D-to-3D human pose lifting methods using GCNs and transformers do not explicitly model the sequential (kinematic chain, hierarchy, body part) dependency between joints.
Approach
Serializes skeleton joints into a sequence that preserves kinematic chain, joint hierarchy, and body-part information (BPS/JHS), and applies a joint-dependent SSM layer, with parameters indexed by joint and a hidden state initialized from the input pose features, alongside GCN and Transformer modules.
Key results
- Achieves 48.1mm P1 and 37.9mm P2 with CPN 2D inputs on Human3.6M, outperforming prior methods.
- Achieves 31.3mm P1 with ground-truth 2D inputs on Human3.6M, best among deterministic methods (1.1mm better than Chen et al. 2024).
- On MPI-INF-3DHP, PCK 86.7 and AUC 54.5, second in AUC behind Zhai et al. 2023 (57.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the pose SSM module to the GCN-Transformer estimator (ablation) | GCN-Transformer pose estimator · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.7 -> 48.1 | Helped |
| replace a GCN module with the pose SSM module (ablation) | GCN-GCN-Transformer pose estimator · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.9 -> 48.1 | Helped |
| replace a Transformer module with the pose SSM module (ablation) | GCN-Transformer-Transformer pose estimator · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.8 -> 48.1 | Helped |
| use joint hierarchy-sorted serialization instead of body part-sorted serialization (ablation) | Pose SSM module · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.5 -> 48.1 | Helped |
| use joint hierarchy-sorted serialization instead of body part-sorted serialization (ablation) | Pose SSM module · Human3.6M, CPN | P-MPJPE (mm), Protocol #2: 38.1 -> 37.9 | Helped |
| make SSM parameters depend on both joints and poses instead of being shared across joints and poses (ablation) | Pose SSM layer · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.6 -> 48.3 | Helped |
| make SSM parameters joint-dependent but shared across poses, rather than dependent on both joints and poses (ablation) | Pose SSM layer · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.3 -> 48.2 | Helped |
| initialize the hidden state from input pose features instead of zero (ablation) | Joint-dependent SSM layer · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.2 -> 48.1 | Helped |
| initialize the hidden state from input pose features instead of zero (ablation) | Joint-dependent SSM layer · Human3.6M, CPN | P-MPJPE (mm), Protocol #2: 38.0 -> 37.9 | Helped |
| use the proposed estimator instead of the strongest listed baseline | 3D pose estimation · Human3.6M, CPN | MPJPE (mm), Protocol #1: 48.5 -> 48.1 | Helped |
| use the proposed estimator instead of the strongest listed baseline | 3D pose estimation · Human3.6M, GT 2D poses | MPJPE (mm), Protocol #1: 31.6 -> 31.3 | Helped |
| use the proposed estimator instead of the strongest listed baseline | 3D pose estimation · MPI-INF-3DHP test, All | PCK: 87.2 -> 86.7 | Hurt |
Surprising findings
- Learnable parameters shared across poses but varying per joint performed better than pose-dependent parameters as in Mamba, suggesting pose-invariance helps for pose data.
Limitations
- Struggles in crowded and occluded scenarios.
- Difficult actions such as sitting down and photoing remain challenging.
- Performance on the two evaluated datasets is saturated, so more challenging datasets are needed.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP
Benchmarks: Human3.6M MPI-INF-3DHP
Metrics: MPJPE P-MPJPE PCK AUC
Baselines: GraFormer HTNet HopFIR MotionBERT-style lifting baselines (Martinez et al. 2017) Chen et al. 2024 Gong et al. 2023 (DiffPose) Zou and Tang 2021
Builds on: Mamba S4 DC-GCT Modulated GCN
Closest papers by meaning
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.90)
- PoseMamba: Monocular 3D Human Pose Estimation with Bidirectional Global-Local Spatio-Temporal State Space Model (AAAI 2025 · similarity 0.90)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.87)
- Video-Based Human Pose Regression via Decoupled Space-Time Aggregation (CVPR 2024 · similarity 0.87)
- ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.87)
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.