Research map / Detection and segmentation / CNN features
Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a recurrent deformable-attention framework that mines fine-grained, structure-aware features from support and query images, plus keypoint mixup padding, for category-agnostic pose estimation on MP-100.
Problem
Category-agnostic pose estimation must locate keypoints on query images for novel classes from few annotated support images; existing methods use coarse heatmap pooling and cross-attention, and zero-pad keypoints to align numbers across classes, which gives weak pixel-level localization and sparse s
Approach
Deformable attention whose reference points are keypoints (fine-grained) and offset along keypoint link graphs via BFS (structure-aware), applied recurrently to both support and query features, combined with mixup-generated padded keypoints replacing zero padding.
Key results
- Achieves 78.72 mPCK on MP-100 Split1 in 1-shot, versus 77.53 for SCAPE (prior SOTA), and 73.42 average over 5 splits.
- Improves PCK@0.05 by +3.2% over SCAPE on Split1 (57.30 vs 54.09).
- Ablation: recurrent support mining adds +3.35 mPCK; FGSA structure-aware reference points add +3.05; mixup padding and mixup loss add further gains to 78.72.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use recurrent FGSA feature mining with deformable attention | CAPE model · MP-100 Split1 1-shot | mPCK: 77.53 -> 78.72 | Helped |
| use recurrent support feature mining (ablation) | Base model · MP-100 Split1 1-shot | mPCK: 69.82 -> 73.18 (+3.35% mPCK) | Helped |
| use link-based reference points to mine structure-aware features (ablation) | Recurrent feature mining model · MP-100 Split1 1-shot | mPCK: 73.18 -> 76.23 (+3.05% mPCK) | Helped |
| add keypoint mixup padding (ablation) | Link-based FGSA model · MP-100 Split1 1-shot | mPCK: 76.23 -> 77.41 | Helped |
| add the mixup-keypoint supervision loss (ablation) | Mixup-padded model · MP-100 Split1 1-shot | mPCK: 77.41 -> 78.72 | Helped |
| replace the recurrent pipeline with stacked support-feature layers followed by stacked keypoint-feature layers (ablation) | Full model · MP-100 Split1 | mPCK: 78.72 -> 77.48 | Hurt |
| replace class-wise links with fully connected links (ablation) | Full model · MP-100 Split1 | mPCK: 78.72 -> 76.17 | Hurt |
| replace class-wise links with self-links only (ablation) | Full model · MP-100 Split1 | mPCK: 78.72 -> 75.23 | Hurt |
| use random mixup padding instead of uniform padding at test time (ablation) | Full model · MP-100 Split1 | mPCK: 78.72 -> 78.35 | Hurt |
| use zero padding instead of uniform padding at test time (ablation) | Full model · MP-100 Split1 | mPCK: 78.72 -> 76.61 | Hurt |
| use the full method instead of SCAPE | MP-100 model · MP-100 Split1 1-shot | PCK@0.05: +3.2% on PCK@0.05 | Helped |
Limitations
- Gains over the strongest baseline are modest at coarse thresholds (+1.06 mPCK average on Split1 vs SCAPE)
- Mixup padding at test time (random) slightly lowers performance, so uniform padding is used instead
Experimental setup
Datasets: MP-100
Benchmarks: MP-100
Metrics: PCK mPCK
Baselines: POMNet CapeFormer ESCAPE MetaPoint GraphCape SCAPE
Builds on: Deformable DETR (deformable attention) CapeFormer Mixup DensePose
Closest papers by meaning
- Meta-Point Learning and Refining for Category-Agnostic Pose Estimation (CVPR 2024 · similarity 0.88)
- ESCAPE: Encoding Super-keypoints for Category-Agnostic Pose Estimation (CVPR 2024 · similarity 0.86)
- CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation (ICLR 2025 · similarity 0.85)
- ProbPose: A Probabilistic Approach to 2D Human Pose Estimation (CVPR 2025 · similarity 0.85)
- Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation (CVPR 2024 · similarity 0.84)
- Detect Any Keypoints: An Efficient Light-Weight Few-Shot Keypoint Detector (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.