Research map / 3D vision / Human pose estimation
TCPFormer: Learning Temporal Correlation with Implicit Pose Proxy for 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TCPFormer, which uses a learnable implicit pose proxy as an intermediate representation with cross-attention update/invocation and a proxy attention fusion to learn richer temporal correlations for 3D human pose lifting.
Problem
Existing multi-frame 3D human pose lifting transformers establish only a single 1-to-T temporal mapping per pose, so accuracy gains saturate as the input frame count grows.
Approach
Introduce a trainable implicit pose proxy (length L, Gaussian-initialized) that is updated from pose features via cross-attention (PUM), invoked back to enhance the pose sequence (PIM), and whose two cross-attention matrices are composed into an aggregation matrix that is adaptively fused with self-attention (PAM), so each proxy builds its own 1-to-T temporal mapping.
Key results
- On Human3.6M with 243 frames (2D GT input), TCPFormer reaches 15.5mm MPJPE; with detected 2D input it reaches 37.9mm MPJPE and 31.7mm P-MPJPE, versus 39.2mm for MotionBERT.
- On MPI-INF-3DHP with 81 frames, TCPFormer achieves PCK 99.0%, AUC 87.7%, MPJPE 15.0mm.
- Ablation on Human3.6M: adding PUM, PIM, and PAM reduces MPJPE from 42.2mm (proxy only) to 37.9mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PUM to the proxy-only model (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 42.2 -> 39.5 | Helped |
| Add PIM to the proxy + PUM model (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 39.5 -> 38.7 | Helped |
| Add PAM to the proxy + PUM + PIM model (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 38.7 -> 37.9 | Helped |
| Set proxy length to 27 instead of 81 (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 38.4 | Hurt |
| Set proxy length to 243 instead of 81 (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 38.6 | Hurt |
| Use random instead of Gaussian proxy initialization (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 39.1 | Hurt |
| Use Laplacian instead of Gaussian proxy initialization (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 38.2 | Hurt |
| Replace cross attention with MLP in both PIM and PUM (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 38.7 -> 40.8 | Hurt |
| Use MLP in PUM instead of cross attention (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 38.7 -> 39.6 | Hurt |
| Use MLP in PIM instead of cross attention (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 38.7 -> 39.5 | Hurt |
| Use fixed fusion with range 0 instead of trainable fusion sampled from (0, 1) (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 38.4 | Hurt |
| Use trainable fusion with range 0 instead of sampling from (0, 1) (ablation) | TCPFormer · Human3.6M, T=243 | MPJPE: 37.9 -> 38.2 | Hurt |
Surprising findings
- With 9 input frames on MPI-INF-3DHP, TCPFormer outperforms STCFormer using 81 frames.
Limitations
- Performance gains saturate or drop at proxy length 243 versus 81 in ablation.
- Results on MPI-INF-3DHP and Human3.6M GT input use ground-truth 2D poses, so gains with detected inputs are smaller.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP
Benchmarks: Human3.6M MPI-INF-3DHP
Metrics: MPJPE P-MPJPE PCK AUC
Baselines: MHFormer MixSTE P-STMO STCFormer PoseFormerV2 GLA-GCN MotionBERT KTPFormer
Builds on: PoseFormer MotionBERT temporal consistency loss
Closest papers by meaning
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.88)
- Semantics-aware Test-time Adaptation for 3D Human Pose Estimation (ICML 2025 · similarity 0.86)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.85)
- PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space (NeurIPS 2025 · similarity 0.85)
- PoseIRM: Enhance 3D Human Pose Estimation on Unseen Camera Settings via Invariant Risk Minimization (CVPR 2024 · similarity 0.85)
- Sequential Joint Dependency Aware Human Pose Estimation with State Space Model (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.