Research map / 3D vision / Human pose estimation
KTPFormer: Kinematics and Trajectory Prior Knowledge-Enhanced Transformer for 3D Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KTPFormer, a transformer for 3D human pose estimation whose Q/K/V inputs are enhanced by kinematics (KPA) and joint-trajectory (TPA) prior attention graphs, usable as plug-and-play modules.
Problem
Transformer-based 3D human pose estimation uses plain linear embeddings to form Q, K, V, ignoring known human anatomy and joint motion trajectories, which can cause attention collapse.
Approach
Builds a kinematics topology (fixed skeletal adjacency plus a learnable fully connected global affinity, symmetrized) and a trajectory topology (same-joint temporal links plus learnable global temporal affinity) and applies them as prior-knowledge attention before spatial and temporal MHSA, without altering the transformer structure.
Key results
- Achieves 40.1mm MPJPE on Human3.6M (T=243, CPN inputs) among non-diffusion methods and 19.0mm with ground-truth 2D inputs.
- On MPI-INF-3DHP with T=81 reaches PCK 98.9%, AUC 85.9%, MPJPE 16.7mm, the reported best.
- Adding KPA and TPA to MixSTE reduces MPJPE from 21.6 to 19.0mm on Human3.6M with only +0.02M parameters and +21M FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use KTPFormer as the diffusion denoiser instead of D3DP | D3DP · Human3.6M (CPN 2D, T=243) | MPJPE (mm): 35.4 -> 33.0 | Helped |
| use KTPFormer as the diffusion denoiser instead of D3DP | D3DP · Human3.6M (CPN 2D, T=243) | P-MPJPE (mm): 28.7 -> 26.2 | Helped |
| use KTPFormer as the diffusion denoiser instead of D3DP | D3DP · Human3.6M (GT 2D, T=243) | MPJPE (mm): 18.4 -> 18.1 | Helped |
| use KTPFormer | 3D pose estimator · MPI-INF-3DHP test (T=81) | PCK: 98.7 -> 98.9 | Helped |
| use KTPFormer | 3D pose estimator · MPI-INF-3DHP test (T=81) | AUC: 83.9 -> 85.9 | Helped |
| use KTPFormer | 3D pose estimator · MPI-INF-3DHP test (T=81) | MPJPE (mm): 23.1 -> 16.7 | Helped |
| use KTPFormer | 3D pose estimator · HumanEva test | MPJPE (mm): 18.9 -> 15.3 | Helped |
| add KPA (ablation) | KTPFormer baseline · Human3.6M (GT 2D, T=243) | MPJPE (mm): 21.8 -> 20.0 | Helped |
| add TPA (ablation) | KTPFormer baseline · Human3.6M (GT 2D, T=243) | MPJPE (mm): 21.8 -> 19.7 | Helped |
| add prior knowledge to KPA and TPA (ablation) | KTPFormer · Human3.6M (GT 2D, T=243) | MPJPE (mm): 21.4 -> 19.0 | Helped |
| add global topologies to KPA and TPA (ablation) | KTPFormer · Human3.6M (GT 2D, T=243) | MPJPE (mm): 20.0 -> 19.0 | Helped |
| add both KPA and TPA (ablation) | KTPFormer baseline · Human3.6M (GT 2D, T=243) | MPJPE (mm): 21.8 -> 19.0 | Helped |
Surprising findings
- Feeding KPA output directly into TPA (united mode) performs worse than the separate mode, attributed to confusing spatial and temporal information.
Limitations
- Diffusion-based result relies on D3DP setup; gains over non-diffusion methods are largest for specific settings only.
- Performance on HumanEva is better at T=27 than T=81, attributed to short sequences, suggesting limited benefit for long contexts on small datasets.
Experimental setup
Datasets: Human3.6M MPI-INF-3DHP HumanEva
Benchmarks: Human3.6M MPI-INF-3DHP HumanEva
Metrics: MPJPE P-MPJPE PCK AUC
Baselines: MixSTE STCFormer PoseFormer MHFormer D3DP DiffPose GLA-GCN P-STMO StridedFormer PoseFormerV2 GraFormer POT
Builds on: PoseFormer MixSTE STCFormer D3DP Transformer
Closest papers by meaning
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.85)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.84)
- PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos (CVPR 2024 · similarity 0.84)
- Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · similarity 0.84)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.84)
- Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.