Research map / Label-efficient and robust learning / Test-time adaptation
Fast Adaptation for Human Pose Estimation via Meta-Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MeTTA, a meta-auxiliary test-time adaptation method for 2D human pose estimation that uses body-specific image inpainting as a self-supervised auxiliary task to adapt pose models in a few fixed inner-loop steps.
Problem
Domain shift makes supervised 2D human pose estimators perform worse on test data than training data; existing test-time adaptation methods need many weight updates, are time-consuming, and cause catastrophic forgetting.
Approach
Combine meta-auxiliary learning with a body-specific image inpainting auxiliary task whose masks are sampled from (ground-truth or predicted) heatmaps, so that inner-loop self-supervised updates are meta-optimized to reduce the primary pose loss, enabling fast, deterministic test-time adaptation.
Key results
- On MPII, MeTTA with HRNet-W32 reaches 91.4 mean PCK, above the listed conventional methods (best 90.5) and TTT (89.4 with ResNet-101).
- On SURREAL to LSP, MeTTA reaches 82.5 PCK@0.05 versus 83.2 for SP+TR, while operating online on single images.
- On Penn Action and Human3.6M with ResNet-50, MeTTA with meta-optimization improves over the SimpleBaseline by +2.37 and +4.74 points, respectively, using K=5 inner updates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add meta-optimization to MeTTA (ablation) | MeTTA · Penn Action | PCK: 86.67 -> 87.60 | Helped |
| add meta-optimization to MeTTA (ablation) | MeTTA · Human3.6M | PCK: 89.50 -> 90.16 | Helped |
| use 5 updates instead of 0 (ablation) | MeTTA · Penn Action | PCK: 85.74 -> 87.60 | Helped |
| use 5 updates instead of 0 (ablation) | MeTTA · Human3.6M | PCK: 87.49 -> 90.16 | Helped |
| use 10 updates instead of 5 (ablation) | MeTTA with meta-optimization · Penn Action | PCK: 87.60 -> 87.71 | Helped |
| use 10 updates instead of 5 (ablation) | MeTTA with meta-optimization · Human3.6M | PCK: 90.16 -> 90.34 | Helped |
| use a middle-sized mask instead of a small mask (ablation) | MeTTA · Penn Action | PCK: 86.80 -> 87.60 | Helped |
| use a large mask instead of a middle-sized mask (ablation) | MeTTA · Penn Action | PCK: 87.60 -> 86.05 | Hurt |
| use a middle-sized mask instead of a small mask (ablation) | MeTTA · Human3.6M | PCK: 89.46 -> 90.16 | Helped |
| use a large mask instead of a middle-sized mask (ablation) | MeTTA · Human3.6M | PCK: 90.16 -> 89.68 | Hurt |
| add a global Perceptual loss to the local MSE loss (ablation) | MeTTA image-inpainting auxiliary task · image reconstruction | reconstruction quality | Helped |
| adapt the source model online at test time | MeTTA · SURREAL → LSP | PCK@0.05 All: 82.5 vs 83.2 | Mixed |
Surprising findings
- Meta-optimization with K=5 inner steps (62 GFLOPs) gives better accuracy than non-meta multi-step adaptation, and K=1 with meta-optimization already beats K=5 without it on Penn Action (86.48 vs 87.12 is not shown; K=1 with MO 86.48 vs K=1 without 86.18).
Limitations
- Requires a joint-training stage and a frozen backbone, so only the Y-shaped head is meta-optimized.
- Gains on SURREAL-to-LSP and SURREAL-to-Human3.6M are not consistently best; MeTTA trails SP+TR on several metrics.
- Mask generation at test time depends on predicted heatmaps, so errors in predictions may propagate to the auxiliary task.
Experimental setup
Datasets: Penn Action Human3.6M MPII SURREAL LSP
Benchmarks: MPII Penn Action Human3.6M LSP
Metrics: PCK PCK@0.05 GFLOPs
Baselines: SimpleBaseline HRNet TTT TTP MMT RegDA-SF POST SP+TR
Builds on: MAML TTP TTT SimpleBaseline HRNet
Closest papers by meaning
- Meta-Auxiliary Learning for Adaptive Human Pose Prediction (AAAI 2023 · similarity 0.91)
- FlexPose: Pose Distribution Adaptation with Limited Guidance (AAAI 2025 · similarity 0.85)
- Semantics-aware Test-time Adaptation for 3D Human Pose Estimation (ICML 2025 · similarity 0.85)
- Optimizing Human Pose Estimation Through Focused Human and Joint Regions (AAAI 2025 · similarity 0.85)
- A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation (CVPR 2024 · similarity 0.85)
- PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.