Research map / 3D vision / Human pose estimation
Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGraFormer, a multi-view 3D human pose estimator built on a deep semantic graph transformer encoder (position, spatial-graph and bone-edge embeddings) with progressive cross-view spatial and multi-view spatial-temporal fusion, trained with MSE only.
Problem
Single-view 3D human pose estimation suffers from depth ambiguity, and existing GCN and transformer methods under-use joint position, spatial structure and bone-edge information and have limited receptive fields for multi-view fusion.
Approach
A semantic graph transformer encoder whose attention is biased by multi-order spatial graph embeddings and which fuses bone-edge features with node features, combined with a progressive cross-view spatial then spatial-temporal fusion that needs no camera parameters.
Key results
- Human3.6M with CPN 2D poses and T=27 reaches 27.6 mm P1 (MPJPE), 8.0 mm better than the best single-view method (Ci et al. 2023) and 2.6 mm better than the best multi-view method without camera parameters (Gordon et al. 2022).
- With ground-truth 2D poses on Human3.6M, MPJPE is 11.7 mm, versus 15.3 mm for Shuai, Wu, and Liu 2022.
- Model has 11.4M parameters and 0.4 GFLOPs at T=27, far fewer than Gordon et al. 2022 (70.4M, 8.5G).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SGraFormer instead of the strongest listed single-view method | Human3.6M 3D pose estimation with detected 2D poses · Human3.6M, detected 2D poses (P1) | MPJPE (mm): 35.6 -> 27.6 | Helped |
| Use SGraFormer instead of Shuai, Wu, and Liu 2022 | Human3.6M 3D pose estimation with ground-truth 2D poses · Human3.6M, ground-truth 2D poses (P1) | MPJPE (mm): 15.3 -> 11.7 | Helped |
| Add spatial and edge feature embeddings to position features (ablation) | SGraFormer · Human3.6M | P1 (mm): 29.4 -> 27.6 | Helped |
| Add spatial feature embedding (ablation) | SGraFormer with position and edge features · Human3.6M | P1 (mm): 28.4 -> 27.6 | Helped |
| Add edge feature embedding (ablation) | SGraFormer with position and spatial features · Human3.6M | P1 (mm): 28.2 -> 27.6 | Helped |
| Add cross-view spatial fusion (CSF) (ablation) | SGraFormer · Human3.6M | P1 (mm): 29.0 -> 27.6 | Helped |
| Add multi-view spatial-temporal fusion (MSTF) (ablation) | SGraFormer · Human3.6M | P1 (mm): 31.7 -> 27.6 | Helped |
| Increase input viewpoints from 1 to 2 (ablation) | SGraFormer · Human3.6M | P1 (mm): 47.8 -> 32.0 | Helped |
| Increase input viewpoints from 2 to 3 (ablation) | SGraFormer · Human3.6M | P1 (mm): 32.0 -> 31.1 | Helped |
| Increase temporal receptive field from 1 to 27 (ablation) | SGraFormer · Human3.6M | P1 (mm): 29.6 -> 27.6 | Helped |
| Increase temporal receptive field from 27 to 243 (ablation) | SGraFormer · Human3.6M | P1 (mm): 27.6 -> 27.3 | Helped |
Surprising findings
- Camera-calibrated multi-view methods such as Iskakov et al. 2019 and Qiu et al. 2019 still outperform the proposed method on Human3.6M.
Limitations
- Performance lags camera-calibrated multi-view methods on Human3.6M.
- Relies on an offline 2D pose estimator, so 2D detection errors propagate.
- Long temporal receptive fields (243 frames) need more data and compute.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP Ski-Pose PTZ-Camera
Benchmarks: Human3.6M MPI-INF-3DHP Ski-Pose PTZ-Camera
Metrics: MPJPE (P1) P-MPJPE (P2) PCK AUC
Baselines: PoseFormer MHFormer Gordon et al. 2022 Shuai, Wu, and Liu 2022 Iskakov et al. 2019 Ci et al. 2023 Zeng et al. 2021 Ionescu et al. 2023
Builds on: Transformer Graph Convolutional Networks CPN
Closest papers by meaning
- Semantic Graph Convolutional Networks for 3D Human Pose Regression (CVPR 2019 · similarity 0.88)
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.88)
- Lifting From the Deep: Convolutional 3D Pose Estimation From a Single Image (CVPR 2017 · similarity 0.88)
- Lifting by Image – Leveraging Image Cues for Accurate 3D Human Pose Estimation (AAAI 2024 · similarity 0.87)
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.86)
- PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.