Research map / Vision-language and multimodal / CLIP and zero-shot
Probabilistic Prompt Distribution Learning for Animal Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PPAP, a probabilistic prompt distribution learning method built on CLAMP/CLIP that models multiple learnable keypoint-attribute prompts as Gaussians for multi-species animal pose estimation.
Problem
Multi-species animal pose estimation suffers from large visual diversity, long-tail data imbalance, and poor cross-species generalization; deterministic single-template text prompts for keypoints fail to capture diverse descriptions and uncertainty.
Approach
Represent each keypoint's learnable prompt set as a factorized multivariate Gaussian (mean from a text decoder, variance from a visual-text decoder), sample prompt representations via reparameterization, add a diversity loss across attribute prompts, and fuse sampled score maps spatially with three strategies.
Key results
- On AP-10K with CLIP ViT-Large, PPAP reaches 78.6 AP vs. CLAMP's 77.8 AP; with ViT-Base it reaches 75.4 AP vs. 74.3.
- On AnimalKingdom P1-All (PCK@0.05), PPAP with ViT-Large achieves 68.4 vs. 65.6 for CLAMP and 66.6 for DARK-HRNet-W32.
- In zero-shot Fishes-to-Birds transfer with ViT-Base, PPAP improves PCK from 0.1 to 12.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PPAP instead of CLAMP | CLIP / ViT-Base animal pose estimation · AP-10K val | AP: 74.3 -> 75.4 | Helped |
| Use PPAP instead of CLAMP | CLIP / ViT-Base animal pose estimation · AnimalKingdom P1-All | PCK@0.05: 62.8 -> 64.5 | Helped |
| Use PPAP instead of CLAMP | ViT-Base animal pose estimation · AnimalKingdom zero-shot: Fishes to Birds | PCK@0.05: 0.1 -> 12.0 | Helped |
| Add Ldiv to the baseline (ablation) | PPAP · AP-10K val | AP: 74.3 -> 74.9 | Helped |
| Add LKL Gaussian-distribution regularization to the baseline (ablation) | PPAP · AP-10K val | AP: 74.3 -> 74.9 | Helped |
| Add both Ldiv and LKL (ablation) | PPAP · AP-10K val | AP: 74.3 -> 75.2 | Helped |
| Add GKP to the model with Ldiv and LKL (ablation) | PPAP · AP-10K val | AP: 75.2 -> 75.4 | Helped |
| Vary the number of attributes Np (ablation) | PPAP · AP-10K val | AP: Np=1: 74.8; Np=2: 75.4; Np=3: 75.1; Np=4: 75.0 | Mixed |
| Vary prompt-loss coefficient β (ablation) | PPAP · AP-10K val | AP: β=1e-4: 74.5; β=5e-5: 74.8; β=1e-5: 75.4; β=1e-6: 74.9 | Mixed |
| Vary feature-loss coefficient γ (ablation) | PPAP · AP-10K val | AP: γ=5e-3: 73.5; γ=5e-4: 75.4; γ=5e-5: 74.7; γ=5e-6: 75.1 | Mixed |
| Vary the number of sampled turns Ns (ablation) | PPAP · AP-10K val | AP: Ns=2: 75.4; Ns=4: 74.7; Ns=6: 74.5; Ns=8: 74.6 | No clear effect |
| Use heuristic, ensemble, or attention-based score-map fusion (ablation) | PPAP · AP-10K val | AP: heuristic selection: 75.1; ensemble selection: 75.4; attention-based selection: 74.6 | Mixed |
Surprising findings
- CLAMP with ViT-Base nearly fails in the Fishes-to-Birds zero-shot transfer (0.1 PCK), which PPAP raises to 12.0.
- With a ResNet-50 backbone, the gain over CLAMP on AP-10K is small (73.4 vs. 72.9 AP), and PPAP is slightly below CLAMP on AP.50.
Limitations
- Gains over CLAMP on ResNet-50 backbone are modest (0.5 AP on AP-10K).
- Hyper-parameters (Np, gamma, beta, Ns) are sensitive and tuned on AP-10K val set.
- Relies on ground-truth bounding boxes in a two-stage pipeline.
Experimental setup
Datasets: AP-10K AnimalKingdom
Benchmarks: AP-10K AnimalKingdom
Metrics: AP AP.50 AP.75 APM APL AR PCK@0.05
Baselines: CLAMP SimpleBaseline HRNet TokenPose TransPose HRFormer ViTPose++ SuperAnimal X-Pose DARK
Builds on: CLAMP CLIP ProDA CoOp PPL
Closest papers by meaning
- UniAP: Towards Universal Animal Perception in Vision via Few-Shot Learning (AAAI 2024 · similarity 0.86)
- Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearance (NeurIPS 2025 · similarity 0.85)
- Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior (AAAI 2024 · similarity 0.85)
- VaMP: Variational Multi-Modal Prompt Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.84)
- SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild (NeurIPS 2026 · similarity 0.84)
- Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.