Research map / 3D vision / Human pose estimation
SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SVTformer, a transformer for multi-view 3D human pose lifting that sequentially decouples spatial, view, and temporal attention in a local-to-global manner, achieving state-of-the-art on Human3.6M, MPI-INF-3DHP and Ski-Pose.
Problem
Multi-view 3D human pose estimation from 2D poses suffers from insufficient modeling of per-view inter-frame joint correspondence and cross-view interactions at each time step.
Approach
Attended spatial-view-temporal patch embedding followed by stacked SVT encoders that alternately apply spatial, view, and temporal attention in sequence (decoupling-coupling-decoupling), without camera parameters or graph priors.
Key results
- Human3.6M with CPN 2D poses (T=27, 4 views): 26.0 mm P1 vs. 26.7 for Zhou et al. 2023 (reported in Table 1 as 26.8 average for Zhou et al.).
- MPI-INF-3DHP: P1 12.0 mm and P2 9.1 mm, versus 16.9/12.1 for SGraFormer.
- Ski-Pose: P1 59.9 mm, versus 62.6 for HMVformer (Zhou et al. 2023).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SVTformer instead of Zhou et al. 2023 | Human3.6M P1 (CPN, T=27) · Human3.6M | MPJPE (mm): 26.8 -> 26.0 | Helped |
| Use SVTformer instead of Zhang et al. 2024 | Human3.6M P1 (GT, T=27) · Human3.6M | MPJPE (mm): 11.7 -> 11.4 | Helped |
| Use SVTformer instead of the strongest listed baseline | MPI-INF-3DHP · MPI-INF-3DHP | PCK: 98.7 -> 99.9 | Helped |
| Use SVTformer instead of the strongest listed baseline | MPI-INF-3DHP · MPI-INF-3DHP | AUC: 90.2 -> 91.6 | Helped |
| Use SVTformer instead of the strongest listed baseline | MPI-INF-3DHP · MPI-INF-3DHP | P1 (mm): 16.9 -> 12.0 | Helped |
| Use SVTformer instead of the strongest listed baseline | MPI-INF-3DHP · MPI-INF-3DHP | P2 (mm): 12.1 -> 9.1 | Helped |
| Use SVTformer instead of the strongest listed baseline | Ski-Pose PTZ-Camera · Ski-Pose PTZ-Camera | P1 (mm): 62.6 -> 59.9 | Helped |
| Use SVTformer instead of the strongest listed baseline | Ski-Pose PTZ-Camera · Ski-Pose PTZ-Camera | P2 (mm): 48.5 -> 47.6 | Helped |
| Replace sequentially alternating modeling with parallel weighted SVT branches (ablation) | SVTformer · Human3.6M | P1 (mm): 26.0 -> 28.7 | Hurt |
| Combine sequential and parallel SVT modeling (ablation) | SVTformer · Human3.6M | P1 (mm): 26.0 -> 26.5 | Hurt |
| Use the parallel-then-sequential SVT variant (ablation) | SVTformer · Human3.6M | P1 (mm): 26.0 -> 27.9 | Hurt |
| Use correspondence order V → S → T instead of S → V → T (ablation) | SVTformer · Human3.6M | P1 (mm): 26.0 -> 29.1 | Hurt |
Surprising findings
- Reordering the sequence to spatial-view-temporal (S→V→T) gives the best result; learning view or temporal first performs worse.
Limitations
- Relies on 2D poses from an external detector (CPN); evaluation on a limited number of datasets.
- Performance drops as the number of fused views decreases (e.g. 35.2 mm with one view).
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP Ski-Pose PTZ-Camera
Benchmarks: Human3.6M MPI-INF-3DHP Ski-Pose PTZ-Camera
Metrics: MPJPE (P1) P-MPJPE (P2) PCK AUC
Baselines: MTF-Transformer HMVformer SGraFormer Gordon et al. 2022 Iskakov et al. 2019 Zhou et al. 2023
Builds on: vanilla transformer encoder CPN
Closest papers by meaning
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.88)
- Multiple View Geometry Transformers for 3D Human Pose Estimation (CVPR 2024 · similarity 0.88)
- ExtPose: Robust and Coherent Pose Estimation by Extending ViTs (ICML 2025 · similarity 0.88)
- Video-Based Human Pose Regression via Decoupled Space-Time Aggregation (CVPR 2024 · similarity 0.88)
- Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation (AAAI 2024 · similarity 0.88)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.