Research map / Architectures and efficiency / Attention and transformers
Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Skeleton-in-Context, a transformer-based in-context learning framework that handles multiple skeleton sequence tasks (motion prediction, pose estimation, joint completion, future pose estimation) with one training run, using task-guided and task-unified prompts.
Problem
In-context learning has not been applied to skeleton sequence modeling; existing MIM-style in-context models from images and point clouds overfit on skeleton sequences (learning only interpolation, causing shrinking) because consecutive poses are highly similar.
Approach
A skeleton-sequence in-context framework with a task-guided prompt (an input-target skeleton pair from a skeleton bank) plus a task-unified prompt (static mean-pose or learnable dynamic pseudo-pose) that replaces random masking, enabling multi-task and unseen-task (motion in-between) generalization.
Key results
- Dynamic-SiC achieves average MPJPE 36.8 on joint completion (3DPW) and 24.4 on motion prediction (AMASS), outperforming multi-task re-structured baselines such as MotionBERT (28.4 MP) and PIC (51.7 MP).
- Dynamic-SiC gets 51.8 MPJPE on pose estimation vs 52.4 for static-TUP ablation and 62.9 on future pose estimation, versus 133.0 for Copy and 91.1 for PIC on joint completion-related rows.
- Cross-dataset generalization: Dynamic-SiC reaches 37.5 MPJPE on AMASS→3DPW motion prediction, better than task-specific EqMotion (50.1) and STCFormer (56.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Dynamic-TUP rather than Static-TUP (ablation) | SiC · AMASS motion prediction | MPJPE (Avg.): 24.6 -> 24.4 | Helped |
| Use Dynamic-TUP rather than Static-TUP (ablation) | SiC · H3.6M pose estimation | MPJPE (Avg.): 51.6 -> 51.8 | Hurt |
| Use Dynamic-TUP rather than Static-TUP (ablation) | SiC · 3DPW joint completion | MPJPE (Avg.): 37.2 -> 36.8 | Helped |
| Use Dynamic-TUP rather than Static-TUP (ablation) | SiC · H3.6M future pose estimation | MPJPE (Avg.): 63.5 -> 62.9 | Helped |
| Add TGP and TUP (ablation) | SiC · AMASS motion prediction | MPJPE: 111.7 -> 24.4 | Helped |
| Add TGP and TUP (ablation) | SiC · 3DPW joint completion | MPJPE: 543.7 -> 36.8 | Helped |
| Add TUP (ablation) | SiC with TGP · AMASS motion prediction | MPJPE: 25.0 -> 24.4 | Helped |
| Add TUP (ablation) | SiC with TGP · H3.6M future pose estimation | MPJPE: 65.2 -> 62.9 | Helped |
| Add TGP (ablation) | SiC with TUP · H3.6M pose estimation | MPJPE: 52.4 -> 51.8 | Helped |
| Add TGP (ablation) | SiC with TUP · H3.6M future pose estimation | MPJPE: 78.4 -> 62.9 | Helped |
| Select prompts randomly rather than by feature similarity or motion similarity (ablation) | SiC · AMASS motion prediction | MPJPE: 25.2 -> 24.4 | Helped |
| Select prompts randomly rather than by feature similarity or motion similarity (ablation) | SiC · H3.6M pose estimation | MPJPE: 52.1 -> 51.8 | Helped |
Surprising findings
- Directly applying MIM-style in-context frameworks (masked pose modeling) to skeleton sequences causes the generated sequence to shrink over time at inference, despite near-perfect reconstruction during training.
- Random prompt selection outperforms feature-similarity or motion-similarity based prompt selection.
Limitations
- Sequence length fixed at 16 frames in experiments.
- Multi-task baselines were re-structured by the authors, so comparisons depend on their reimplementation.
Experimental setup
Datasets: H3.6M AMASS 3DPW
Benchmarks: H3.6M AMASS 3DPW
Metrics: MPJPE N-MPJPE
Baselines: siMLPe EqMotion STCFormer GLA-GCN MotionBERT Copy PointMAE Point-In-Context
Builds on: Painter Point-In-Context MotionBERT Masked Autoencoders
Closest papers by meaning
- Motif Guided Graph Transformers with Combinatorial Skeleton Prototype Learning for Skeleton-Based Person Re-Identification (AAAI 2025 · similarity 0.83)
- Universal Skeleton Understanding via Differentiable Rendering and MLLMs (ICML 2026 · similarity 0.83)
- Heterogeneous Skeleton-Based Action Representation Learning (CVPR 2025 · similarity 0.82)
- Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.81)
- Less is More: Compact-Token Masked Feature Learning for Skeleton Representation Learning (NeurIPS 2026 · similarity 0.81)
- SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.