Research map / Vision-language and multimodal / Human and embodied modeling

Less is More: Compact-Token Masked Feature Learning for Skeleton Representation Learning

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Vision-language pretraining

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SLiM, a compact-token skeleton self-supervised framework that combines decoder-free teacher-guided masked feature prediction with global-local contrastive learning, plus Semantic Tube Masking and Skeleton-Aware Augmentations, achieving SOTA linear-eval accuracy with 7.89x lower downstream encoder inference GFLOPs.

Problem

Skeleton self-supervised learning with masked auto-encoders relies on dense skeletal-temporal tokens and heavy coordinate-reconstruction decoders, causing high pre-training cost and expensive downstream inference; contrastive methods miss fine-grained motion detail.

Approach

Replace coordinate reconstruction with decoder-free, EMA-teacher-guided masked feature prediction on a compact token grid (8x25), unify it with global-local contrastive learning in a shared encoder, and prevent interpolation shortcuts via anatomically grouped tube masking and bone-aware augmentations.

Key results

• Linear evaluation: 87.9% NTU-60 X-Sub, 93.2% NTU-60 X-View, 81.2% NTU-120 X-Sub, 83.6% NTU-120 X-Set, best on four of five protocols (AMR second-best on PKU-MMD II by 0.6 points).
• Encoder inference cost of 3.59 GFLOPs with an 8x25 token grid versus 28.32 GFLOPs for dense 30x25 MAE baselines, a 7.89x reduction.
• Semi-supervised 1% label setting improves the previous best by 0.3 (X-Sub) and 2.9 (X-View) points.

Surprising findings

• Standard MAE baselines degrade substantially when the temporal token budget is reduced (e.g., MAMP drops from 87.00% to 80.90%), while SLiM stays near 90.5-90.9% across 8, 15, and 30 temporal tokens.

Limitations

• Evaluated only on 3D joint modality; multi-stream fusion and noisy 2D poses, missing joints, and heterogeneous topologies are left for future work.
• Main results use a single pre-training run; reported variability reflects linear probes rather than pre-training seeds.
• Efficiency comparison of total pre-training cost is not controlled, since methods differ in view counts and pipelines.

Experimental setup

Datasets: NTU RGB+D 60 NTU RGB+D 120 PKU-MMD II

Benchmarks: NTU RGB+D 60 NTU RGB+D 120 PKU-MMD II

Metrics: Top-1 accuracy GFLOPs k-NN top-1 retrieval accuracy

Baselines: AMR GFP S-JEPA MAMP SkeletonMAE MacDiff IGM USDRL HSP

Builds on: DINO MAE MAMP GFP AimCLR

Closest papers by meaning

• MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning (CVPR 2024 · similarity 0.89)
• USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelation (AAAI 2025 · similarity 0.87)
• SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology (NeurIPS 2026 · similarity 0.86)
• Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.86)
• Universal Skeleton Understanding via Differentiable Rendering and MLLMs (ICML 2026 · similarity 0.86)
• Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.86)