Research map / Probabilistic and scientific modeling / Molecules and proteins
Skeleton-based Action Recognition with Non-linear Dependency Modeling and Hilbert-Schmidt Independence Criterion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Gaussian-correlation dependency refinement of skeleton graphs plus an HSIC-based classification objective with auxiliary-model distillation and a multi-stream ensemble for skeleton action recognition.
Problem
Skeleton-based action recognition methods model only physically connected joints and estimate class probability densities in high-dimensional Euclidean space, limiting accuracy on distant-joint dependencies and high-dimensional motion features.
Approach
Models dependencies between every pair of joints via a Gaussian correlation with tunable kernel width to refine the skeleton adjacency, and classifies actions by HSIC between Matérn-kernel features of auxiliary-enhanced motion features and labels, with multi-kernel-width ensembles.
Key results
- Achieves 93.7% X-Sub and 97.3% X-View on NTU RGB+D 60, exceeding the prior best of 93.0% and 97.1%.
- Achieves 90.6% X-Sub and 91.7% X-Set on NTU RGB+D 120 with 4-stream ensemble, surpassing FR-Head by 1.1% and 0.8% under the same ensemble setup.
- Reaches 97.2% top-1 on Northwestern-UCLA, above prior best of 97.0% (InfoGCN).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of SkeletonGCL | Skeleton-based action recognition · NTU RGB+D 120 X-Sub | Top-1 accuracy: 89.8 -> 90.6 | Helped |
| Use the proposed method instead of InfoGCN | Skeleton-based action recognition · NTU RGB+D 120 X-Set | Top-1 accuracy: 91.2 -> 91.7 | Helped |
| Use the proposed method instead of InfoGCN | Skeleton-based action recognition · NTU RGB+D 60 X-Sub | Top-1 accuracy: 93.0 -> 93.7 | Helped |
| Use the proposed method instead of InfoGCN | Skeleton-based action recognition · NTU RGB+D 60 X-View | Top-1 accuracy: 97.1 -> 97.3 | Helped |
| Use the proposed method instead of InfoGCN | Skeleton-based action recognition · Northwestern-UCLA | Top-1 accuracy: 97.0 -> 97.2 | Helped |
| Add joint and bone inputs rather than using joint input only (ablation) | Ours · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 89.3 | Helped |
| Add four ensemble streams rather than using joint input only (ablation) | Ours · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 90.6 | Helped |
| Add an auxiliary model (ablation) | Base model · NTU RGB+D 120 X-Sub, joint input | accuracy: 83.4 -> 86.0 | Helped |
| Add the HSIC loss (ablation) | Ours · NTU RGB+D 120 X-Sub, joint input | accuracy: 85.1 -> 86.0 | Helped |
| Add the distillation loss (ablation) | Ours · NTU RGB+D 120 X-Sub, joint input | accuracy: 85.2 -> 86.0 | Helped |
| Add both HSIC and distillation losses (ablation) | Ours · NTU RGB+D 120 X-Sub, joint input | accuracy: 84.9 -> 86.0 | Helped |
| Add the HSIC-based learning objectives (ablation) | Baseline · NTU RGB+D 120 X-Sub, joint input | accuracy: +1.3 | Helped |
Limitations
- Few-shot and unsupervised learning not explored
- Gains over prior state of the art are small (e.g., 0.9% on NTU60 X-Sub)
- Ablations only on NTU RGB+D 120 X-Sub joint modality
Experimental setup
Datasets: NTU RGB+D 60 NTU RGB+D 120 Northwestern-UCLA
Benchmarks: NTU RGB+D 60 NTU RGB+D 120 Northwestern-UCLA
Metrics: Top-1 accuracy
Baselines: ST-LSTM GCA-LSTM ST-GCN 2s-AGCN DC-GCN+ADG MS-G3D Dynamic GCN CTR-GCN MST-GCN Ta-CNN EfficientGCN-B4 STF
Builds on: ST-GCN CTR-GCN InfoGCN HSIC-InfoGAN
Closest papers by meaning
- Heterogeneous Skeleton-Based Action Representation Learning (CVPR 2025 · similarity 0.91)
- InfoGCN: Representation Learning for Human Skeleton-Based Action Recognition (CVPR 2022 · similarity 0.90)
- Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.89)
- Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition (CVPR 2025 · similarity 0.88)
- Recovering Complete Actions for Cross-dataset Skeleton Action Recognition (NeurIPS 2024 · similarity 0.88)
- MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.