Research map / Detection and segmentation / Open-set and few-shot detection
Detect Any Keypoints: An Efficient Light-Weight Few-Shot Keypoint Detector
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a lightweight few-shot keypoint detector (lwFSKD) that fuses keypoint modulation and detection via non-linear kernel generation and non-parametric detection, plus mean feature based contrastive learning to reduce domain shift across species.
Problem
Few-shot keypoint detection (FSKD) for unseen animal species: existing modulation-detection separated designs become heavy, slow and memory-hungry as the number of prompted keypoints grows, and suffer from large domain shift between seen and unseen species.
Approach
Replace per-keypoint modulated feature maps with conditionally generated multi-resolution convolution kernels (non-linear kernel generator) applied directly to the query feature map for simultaneous modulation and detection, and align episode-level mean keypoint features across episodes with a contrastive loss (MFCL).
Key results
- 1-shot novel keypoint detection on Animal pose reaches 60.92 mean PCK with S=(1,3) kernels versus 56.04 for FSKD-D (+5.62 over FSKD-D as reported in the efficiency table), with 27.6M vs 65.2M parameters.
- Inference time per query image drops from 4.13 ms (FSKD) to 1.86 ms (lwFSKD S=1,3) measured on a V100.
- 5-shot novel keypoint detection on Animal pose improves to 57.61 PCK versus 54.63 for FSKD-D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use lwFSKD with simultaneous modulation and detection | FSKD-D · Animal pose, 1-shot novel keypoints | PCK: 44.75 -> 50.37 | Helped |
| Use lwFSKD with simultaneous modulation and detection | FSKD-D · AwA, 1-shot novel keypoints | PCK: 64.76 -> 69.81 | Helped |
| Use lwFSKD with simultaneous modulation and detection | FSKD-D · CUB, 1-shot novel keypoints | PCK: 77.89 -> 83.34 | Helped |
| Use lwFSKD with simultaneous modulation and detection | FSKD-D · NABird, 1-shot novel keypoints | PCK: 56.04 -> 60.92 | Helped |
| Add the upsampling module U (ablation) | Baseline (⋄) · Animal pose, 1-shot novel keypoints | PCK: 39.82 -> 41.71 | Helped |
| Add a linear kernel generator with U (ablation) | Baseline + U · Animal pose, 1-shot novel keypoints | PCK: 41.71 -> 43.01 | Helped |
| Replace the linear kernel generator with a non-linear kernel generator (ablation) | Baseline + U + linear KG · Animal pose, 1-shot novel keypoints | PCK: 43.01 -> 46.04 | Helped |
| Add instance-based contrastive learning (ablation) | Baseline + U + non-linear KG · Animal pose, 1-shot novel keypoints | PCK: 46.04 -> 47.63 | Helped |
| Replace instance-based CL with MFCL without negative keypoints (ablation) | Baseline + U + non-linear KG + instance CL · Animal pose, 1-shot novel keypoints | PCK: 47.63 -> 49.39 | Helped |
| Add negative keypoints to MFCL (ablation) | Baseline + U + non-linear KG + MFCL† · Animal pose, 1-shot novel keypoints | PCK: 49.39 -> 50.37 | Helped |
| Use kernel resolutions S=1 and S=3 together instead of S=1 alone (ablation) | lwFSKD · Animal pose, 1-shot novel keypoints | PCK: 49.22 -> 50.37 | Helped |
| Use kernel resolutions S=1 and S=3 together instead of S=3 alone (ablation) | lwFSKD · Animal pose, 1-shot novel keypoints | PCK: 48.54 -> 50.37 | Helped |
Surprising findings
- Using a single kernel resolution (S=1) already achieves most of the gain, and multi-resolution kernels with S=5 are slower than S=1,3 in inference time.
Limitations
- Gains on the Animal pose dataset are reported on a single backbone (ResNet50) and modest kernel-resolution sweeps.
- Negative keypoint control introduces hyperparameters (alpha, rho, Nneg) that must be set manually.
Experimental setup
Datasets: Animal pose dataset AwA CUB NABird
Benchmarks: Animal pose dataset AwA CUB NABird
Metrics: PCK parameters inference time training time GPU memory
Baselines: ProtoNet RelationNet LwoF ProbIntr FSKD-R FSKD-D
Builds on: FSKD (Lu and Koniusz 2022) LwoF HigherHRNet supervised contrastive learning
Closest papers by meaning
- Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features (CVPR 2024 · similarity 0.84)
- Recurrent Feature Mining and Keypoint Mixup Padding for Category-Agnostic Pose Estimation (CVPR 2025 · similarity 0.84)
- Incremental Object Keypoint Learning (CVPR 2025 · similarity 0.83)
- ESCAPE: Encoding Super-keypoints for Category-Agnostic Pose Estimation (CVPR 2024 · similarity 0.82)
- Meta-Point Learning and Refining for Category-Agnostic Pose Estimation (CVPR 2024 · similarity 0.82)
- Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.