Research map / Label-efficient and robust learning / Recognition and long-tail
FlexPose: Pose Distribution Adaptation with Limited Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
FlexPose adapts a pre-trained StyleGAN skeleton-image pose generator to a new pose distribution by fine-tuning a few sparse block-diagonal linear layers on a handful of target poses, using Pose-mixup regularization.
Problem
Annotating human poses for newly collected images is costly, and pose annotation generators trained on one dataset do not transfer well to another pose distribution when only a few target annotations are available.
Approach
Treat pose annotations as skeleton images, learn a generic StyleGAN pose prior from a source dataset, and transfer it to the target by calibrating only a few early-layer affine style transformations with linear, sparse, and Pose-mixup regularizations, avoiding any image-domain adaptation.
Key results
- On COCO→Human3.6M, FlexPose reaches MMD² 0.029 and FD 0.80, beating LoRA (0.035, 1.36), AdaGAN and FreezeD.
- On 3DHP→Simplified Human3.6M, FlexPose improves MSE from 12.66 (baseline) to 5.98 and PCK10 from 0.000 to 0.467.
- On 300-VW→WFLW face landmarks, FlexPose achieves MSE 11.64 vs 18.78 baseline, comparable to AdaGAN/FreezeD/LoRA (PCK 0.766 vs 0.785 for AdaGAN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt the COCO pose generator to Human3.6M with FlexPose | FlexPose · Human3.6M pose distribution (COCO source) | MMD^2: 0.035 -> 0.029 | Helped |
| adapt the COCO pose generator to Human3.6M with FlexPose | FlexPose · Human3.6M pose distribution (COCO source) | FD: 1.36 -> 0.80 | Helped |
| use FlexPose instead of LoRA for pose-distribution transfer | human pose annotation · Human3.6M pose annotation (COCO source) | PCK: 0.430 -> 0.585 | Helped |
| use FlexPose instead of LoRA for pose-distribution transfer | human pose annotation · Simplified Human3.6M pose annotation, COCO, 3DHP, and SURREAL sources | PCK: 0.512 -> 0.770; 0.314 -> 0.467; 0.337 -> 0.499 | Helped |
| use FlexPose instead of AdaGAN for face-landmark transfer | face landmark annotation · WFLW (300-VW source) | PCK: 0.785 -> 0.766 | Hurt |
| adapt layer 3 only instead of layers 1 and 3 (ablation) | FlexPose · Simplified Human3.6M pose annotation (COCO source; 12 shots) | MSE: 3.82 -> 3.79 | Helped |
| adapt layer 3 only rather than layers 3 and 5 or all layers (ablation) | FlexPose · Simplified Human3.6M pose annotation (COCO source; 12 shots) | MSE: 4.02 -> 3.79; 4.50 -> 3.79 | Helped |
| add Pose-mixup regularization (ablation) | FlexPose · Simplified Human3.6M pose annotation (3DHP source; 12 shots) | MSE: 9.82 -> 5.98 | Helped |
| add Linear regularization (ablation) | FlexPose · Simplified Human3.6M pose annotation (3DHP source; 12 shots) | MSE: 12.32 -> 9.82 | Helped |
| increase target guidance from 12 to 24 or 48 shots (ablation) | FlexPose · Simplified Human3.6M pose annotation (COCO source) | MSE: 3.79 -> 3.80; 3.79 -> 3.73 | No clear effect |
| add COCO to 3DHP as the source dataset (ablation) | FlexPose · Simplified Human3.6M pose annotation (12 shots) | MSE: 5.98 -> 5.28 | Helped |
| add SURREAL to the 3DHP and COCO source datasets (ablation) | FlexPose · Simplified Human3.6M pose annotation (12 shots) | MSE: 5.28 -> 5.19 | Helped |
Surprising findings
- Increasing target shots from 12 to 48 gave no obvious improvement in downstream detector performance, suggesting a strong source prior dominates.
- On the face task FlexPose does not beat AdaGAN on PCK (0.766 vs 0.785), only on MSE tie-level performance.
Limitations
- Target adaptation relies on a few guiding poses; the paper notes mismatch in some regions of the t-SNE embedding.
- Selection of which StyleGAN layer to calibrate is empirical and the best choice is left to future work.
- Performance trade-off between source diversity and similarity to target in multi-source settings.
Experimental setup
Datasets: COCO Human3.6M Simplified Human3.6M MPI-INF-3DHP SURREAL 300-VW WFLW
Benchmarks: Human3.6M Simplified Human3.6M WFLW
Metrics: MMD^2 Frechet distance MSE PCK
Baselines: AdaGAN FreezeD LoRA Baseline (source-trained detector)
Builds on: StyleGAN mixup LoRA KeypointGAN
Closest papers by meaning
- PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation (CVPR 2025 · similarity 0.89)
- PoseGen: Learning to Generate 3D Human Pose Dataset with NeRF (AAAI 2024 · similarity 0.88)
- Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation (NeurIPS 2024 · similarity 0.87)
- HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (CVPR 2025 · similarity 0.86)
- Can Generative Video Models Help Pose Estimation? (CVPR 2025 · similarity 0.86)
- ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.