Research map / 3D vision / Human pose estimation
SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHaRPose, a two-stage coarse-to-fine transformer for 2D human pose estimation that builds high-resolution tokens only for keypoint-related image patches, selected via keypoint-token attention, with a quality predictor deciding whether to run the fine stage.
Problem
Dense high-resolution token representations for transformer-based human pose estimation incur heavy computational cost, even though only sparse keypoint locations need to be detected.
Approach
Use keypoint tokens and a quality token in a plain ViT; mine keypoint-to-patch attention (EMA-accumulated across layers) to select only the relevant patches for re-tokenizing at finer granularity, sharing transformer weights across the coarse and fine stages, and skip the fine stage when the predicted OKS-like quality is high.
Key results
- SHaRPose-Base reaches 77.4 AP on COCO val at 384x288 (vs ViTPose-Base 76.9) with 32.9 GFLOPs vs 44.1.
- On COCO test-dev, SHaRPose-Base achieves 76.7 AP vs ViTPose-Base 76.2, running about 1.4x faster.
- On MPII, SHaRPose-Base reaches 91.4 mean PCKh vs 90.2 for TokenPose-L/D24 and 89.0 for SimpleBaseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHaRPose-Small instead of ViTPose-Small | ViTPose-Small · COCO val, 256×192 | AP: 73.8 -> 74.2 | Helped |
| Use SHaRPose-Base instead of ViTPose-Base | ViTPose-Base · COCO val, 384×288 | AP: +0.5 AP | Helped |
| Use SHaRPose-Base instead of ViTPose-Base | ViTPose-Base · COCO test-dev, 384×288 | AP: +0.5 AP | Helped |
| Increase α from 0.0 to 0.3 (ablation) | SHaRPose-Base · COCO val, 256×192 | AP: 68.4 -> 74.8 | Helped |
| Increase α from 0.3 to 0.4 (ablation) | SHaRPose-Base · COCO val, 256×192 | AP: 74.8 -> 75.5 | Helped |
| Increase α from 0.4 to 0.5 (ablation) | SHaRPose-Base · COCO val, 256×192 | AP: 75.5 -> 75.5 | No clear effect |
| Increase α from 0.3 to 0.5 (ablation) | SHaRPose-Base · COCO val, 384×288 | AP: 77.4 -> 77.5 | No clear effect |
| Add the fine stage (ablation) | SHaRPose coarse-stage model · COCO val, 256×192 | AP: 67.9 -> 75.5 | Helped |
| Add the coarse stage (ablation) | SHaRPose fine-stage model · COCO val, 256×192 | AP: 74.7 -> 75.5 | Helped |
| Add keypoint-related patch selection (ablation) | SHaRPose two-stage model · COCO val, 256×192 | GFLOPs: 24.9 -> 17.1 | Helped |
| Skip the fine stage for more samples by lowering Q_thres (ablation) | SHaRPose-Base · COCO | AP | Hurt |
| Replace coarse-to-fine processing with dynamic token pruning (ablation) | SHaRPose pipeline · COCO val, 256×192 | AP: 75.5 vs 72.7 | Hurt |
Surprising findings
- Pruning-based dynamic transformers (DynamicViT, EViT) lose substantial AP when adapted to pose estimation, indicating pruning limits localization precision.
Limitations
- Quality threshold Q_thres trades accuracy against throughput, requiring application-specific tuning.
- Gains over ViTPose-Base at 256x192 are small (75.5 vs 75.8 AP), with SHaRPose-Base slightly below ViTPose-Base on that setting.
Experimental setup
Datasets: COCO 2017 MPII
Benchmarks: COCO val COCO test-dev MPII val
Metrics: AP AP50 AP75 AR PCKh@0.5 FPS GFLOPs
Baselines: ViTPose-Base HRNet-W48 HRFormer-Base TokenPose-L/D6 SimpleBaseline DynamicViT EViT UDP-HRNet-W48 DARK-HRNet-W48 TokenPose-L/D24 OKDHP
Builds on: ViTPose DynamicViT EViT CF-ViT TokenPose
Closest papers by meaning
- ProbPose: A Probabilistic Approach to 2D Human Pose Estimation (CVPR 2025 · similarity 0.87)
- Optimizing Human Pose Estimation Through Focused Human and Joint Regions (AAAI 2025 · similarity 0.87)
- SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos (AAAI 2025 · similarity 0.86)
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.85)
- ExtPose: Robust and Coherent Pose Estimation by Extending ViTs (ICML 2025 · similarity 0.85)
- FreeMan: Towards Benchmarking 3D Human Pose Estimation under Real-World Conditions (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.