Research map / 3D vision / Human pose estimation
Spatial-Aware Regression for Keypoint Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Spatial-Aware Regression (SAR), which regresses keypoint coordinates and confidence scores from every feature-map grid using spatial location priors, trained with a unified Laplacian-kernel objective, and evaluates it on 2D/3D, whole-body and multi-person keypoint tasks.
Problem
Conventional regression-based keypoint localization discards the spatial location prior through global pooling, which gives inferior accuracy and prevents localizing multiple keypoints of the same type (e.g., multi-person pose estimation).
Approach
Regress coordinates from each grid of the feature map (offset from the grid's own location), jointly predict a confidence score per grid, and optimize both with a single objective that maximizes the confidence-weighted regression quality (Laplacian-kernel score), removing heuristic grid selection and multi-task hyperparameters.
Key results
- SAR reaches 72.5 AP on COCO val at 256x192 with ResNet-50 (SimplePose), vs 53.8 for conventional regression and 70.7 for heatmap.
- SAR with HRNet-W48 achieves 76.3 AP on COCO test-dev at 384x288, above heatmap SimCC (76.0) and RLE (75.7).
- On Human3.6M, SAR reduces MPJPE from 48.6 (RLE-style baseline) to 47.1 with flip test; on CrowdPose, SAR improves DEKR AP from 66.0 to 67.0 on the multi-person benchmark.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spatial location priors and per-grid regression with selection | SimplePose, ResNet-50 · COCO Keypoint val | AP: 53.8 -> 72.5 | Helped |
| Use SAR instead of heatmap localization | SimplePose, ResNet-50 · COCO Keypoint val | AP: 70.7 -> 72.5 | Helped |
| Generate grid-wise features from pooled features with an FC layer (ablation) | SAR · COCO Keypoint val | AP: 72.5 -> 71.1 | Hurt |
| Add SAR to HRNet-W48 | COCO Keypoint test-dev · COCO Keypoint test-dev | AP: 76.0 -> 76.3 | Helped |
| Use SAR for whole-body keypoint localization | HRNet-W48 · COCO Whole-Body | whole-body AP: 65.4 -> 66.6 | Helped |
| Use SAR instead of heatmap localization with AE | COCO bottom-up MPPE · COCO bottom-up MPPE | AP: 48.4 -> 50.0 | Helped |
| Replace CenterNet keypoint regression with SAR | CenterNet · COCO single-stage MPPE | AP: 57.6 -> 60.3 | Helped |
| Replace DEKR keypoint regression with SAR | DEKR · OCHuman single-stage MPPE | AP: 35.6 -> 40.9 | Helped |
| Replace DEKR keypoint regression with SAR | DEKR · CrowdPose single-stage MPPE | AP: 65.2 -> 66.3 | Helped |
| Add SAR for 3D keypoint localization | Human3.6M regression baseline · Human3.6M | MPJPE: 51.7 -> 47.1 | Helped |
| Use a 3D feature map for spatial location prior (ablation) | SAR · Human3.6M | MPJPE: 47.1 -> 49.4 | Hurt |
| Decouple depth estimation into a separate localization branch (ablation) | SAR · Human3.6M | MPJPE: 47.1 -> 48.9 | Hurt |
Surprising findings
- The ablation with only the pooled feature plus a spatial-location FC (f^g+FC) reaches 71.1 AP, showing most of the gain comes from the location prior rather than visual features.
Limitations
- Lambda hyperparameter (set to 16) and the output-selection threshold are still required; the method relies on a confidence threshold at inference.
- Results on 3D pose lag the Integral Pose baseline (by 1.2 and 1.7 MPJPE), and decoupled 3D variant depends on extra depth branch.
Experimental setup
Datasets: COCO Keypoint MPII COCO Whole-Body OCHuman CrowdPose Human3.6M
Benchmarks: COCO Keypoint MPII COCO Whole-Body OCHuman CrowdPose Human3.6M
Metrics: OKS-based AP AR PCKh MPJPE
Baselines: Heatmap (SimplePose) Heatmap+Offset Direct regression RLE SimCC HRNet Integral Pose ZoomNAS ZoomNet DeepPose
Builds on: SimplePose AE associative embedding CenterNet DEKR Integral pose regression
Closest papers by meaning
- ProbPose: A Probabilistic Approach to 2D Human Pose Estimation (CVPR 2025 · similarity 0.85)
- Where Am I and What Will I See: An Auto-Regressive Model for Spatial Localization and View Prediction (ICLR 2025 · similarity 0.83)
- Scene-agnostic Pose Regression for Visual Localization (CVPR 2025 · similarity 0.83)
- R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization (CVPR 2025 · similarity 0.83)
- Learning to Produce Semi-dense Correspondences for Visual Localization (CVPR 2024 · similarity 0.83)
- Generalizable Object Keypoint Localization from Generative Priors (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.