Research map / Label-efficient and robust learning / Recognition and long-tail
ESCAPE: Encoding Super-keypoints for Category-Agnostic Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ESCAPE, a Bayesian category-agnostic pose estimation method that models keypoint prototypes with a mixture of super-keypoints learned from base categories, matched to novel keypoints via a learnable SuperGlue-style network, without keypoint identifiers.
Problem
Category-agnostic pose estimation (CAPE) predicts keypoints for novel object categories from few annotated support samples; local matching overfits to sparse support, and keypoint identifiers are impractical in open-world settings.
Approach
Place a Bayesian prior over keypoint features as a mixture of super-keypoints (clusters of semantically related keypoints across base categories); adapt to novel categories with EM, where a learnable matching network (self/cross-attention + Sinkhorn) assigns novel keypoints to super-keypoints, and the MAP prototype pulls support means toward matched super-keypoints.
Key results
- ESCAPE outperforms CapeFormer (without keypoint identifiers) by about 2.5% in 1-shot (82.74 vs 80.75 mean PCK) and about 0.5% in 5-shot on MP-100.
- Learnable matching network improves over simple matching (MAP) by about 3% PCK (86.89 vs 83.88 in the ablation).
- Integrating ESCAPE with DINOv2 raises PCK from 88.31 to 89.14.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ESCAPE instead of CapeFormer | Category-agnostic pose estimation · MP-100 1-shot | Mean PCK@0.2: 80.75 -> 82.74 | Helped |
| Use ESCAPE instead of CapeFormer | Category-agnostic pose estimation · MP-100 5-shot | Mean PCK@0.2: 87.19 -> 87.63 | Helped |
| Add a super-keypoint prior using simple matching (ablation) | MLE prototype estimation · MP-100 | PCK: 83.00 \pm 0. 16 -> 83.88 \pm 0. 15 | Helped |
| Replace simple matching with the learnable matching network (ablation) | MAP prototype estimation · MP-100 | PCK: 83.88 \pm 0. 15 -> 86.89 \pm 0. 14 | Helped |
| Use ESCAPE prototype estimation instead of MLE (ablation) | DIFT features · MP-100 | PCK: 60.61 -> 62.47 | Helped |
| Use ESCAPE prototype estimation instead of MLE (ablation) | DINOv2 features · MP-100 | PCK: 88.31 -> 89.14 | Helped |
Surprising findings
- A simple MLE-based ResNet-50 baseline is comparable to POMNet, showing that the overfitting problem is shared by prototype-matching methods.
Limitations
- Relies on ground-truth bounding boxes to crop objects, following prior protocol.
- Matching network and super-keypoint discovery depend on base-category keypoint features from the trained extractor.
- Gains over CapeFormer are small in the 5-shot setting.
Experimental setup
Datasets: MP-100
Benchmarks: MP-100
Metrics: PCK@0.2
Baselines: ProtoNet MAML Finetune POMNet CapeFormer DIFT DINOv2
Builds on: POMNet SuperGlue density peaks clustering DIFT Sinkhorn algorithm
Closest papers by meaning
- Meta-Point Learning and Refining for Category-Agnostic Pose Estimation (CVPR 2024 · similarity 0.92)
- CapeX: Category-Agnostic Pose Estimation from Textual Point Explanation (ICLR 2025 · similarity 0.90)
- SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.87)
- Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation (CVPR 2024 · similarity 0.87)
- Unsupervised 3D Structure Inference from Category-Specific Image Collections (CVPR 2024 · similarity 0.87)
- Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.