Research map / Architectures and efficiency / CNN design
Are Spatial-Temporal Graph Convolution Networks for Human Action Recognition Over-Parameterized?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows ST-GCNs for skeleton-based action recognition are over-parameterized via lottery-ticket experiments, proposes a sparse ST-GCN generator with fixed random masks and a group-lasso warm-up, and assembles multi-sparsity models that beat dense baselines with fewer parameters.
Problem
ST-GCNs for skeleton-based human action recognition show similar accuracy across variants and are large; the question is whether they are over-parameterized and whether sparse subnetworks can match dense performance at lower cost.
Approach
Applies the lottery ticket hypothesis to ST-GCNs and proposes a sparse ST-GCN generator that trains from random initialization using a fixed random mask, a group-lasso penalty warm-up that pushes masked weights toward zero, and then fine-tunes only the kept weights; multi-sparsity copies are ensembled to improve accuracy.
Key results
- x
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| assemble sparse backbones at sparsity levels 0.6, 0.8, 0.95, and 0.99 | DG-STGCN · NTU60-Xsub | top-1 accuracy: 93.2 -> 92.9 | Helped |
| assemble sparse backbones at sparsity levels 0.6, 0.8, 0.95, and 0.99 | DG-STGCN · NTU120-Xsub | top-1 accuracy: 89.6 -> 90.4 | Helped |
| add the information-compression penalty loss during warm-up (ablation) | sparse ST-GCN generator · NTU120-Xset | top-1 accuracy | Helped |
| remove the warm-up stage (ablation) | sparse ST-GCN generator · NTU RGB+D | top-1 accuracy | Hurt |
| extract a sparse subnetwork from a fully trained dense network, masking out 80% of parameters (ablation) | ST-GCN and CTR-GCN · NTU RGB+D benchmarks | top-1 accuracy | No clear effect |
| increase sparsity to 0.99 (ablation) | DG-STGCN · NTU60-Xview | top-1 accuracy: 96.4 -> 94.9 | Hurt |
| increase sparsity to 0.99 (ablation) | DG-STGCN · FineGYM | top-1 accuracy: 95.1o -> 92.2 | Hurt |
Surprising findings
- Without warm-up, adaptive-graph ST-GCNs (CTR-GCN, DG-STGCN) collapse at sparsity above 0.8, while the warm-up stage fixes this.
- Optimal sparsity level increases with model size and decreases with dataset size.
Limitations
- Sparsity level is manually set rather than learned.
- Multi-level sparse model is assembled after separate single-model training rather than trained end-to-end.
- Accuracy degrades noticeably on NTU120 and at 99% sparsity for several backbones.
Experimental setup
Datasets: NTU RGB+D 60 NTU RGB+D 120 Kinetics-400 FineGYM
Benchmarks: NTU RGB+D 60 NTU RGB+D 120 Kinetics-400 FineGYM
Metrics: top-1 accuracy parameter count
Baselines: ST-GCN AA-GCN CTR-GCN DG-STGCN MS-G3D PoseConv3D ST-GCN++ SKE2GRID
Builds on: lottery ticket hypothesis ST-GCN CTR-GCN DG-STGCN UGS
Closest papers by meaning
- Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition (AAAI 2021 · similarity 0.92)
- Dynamic Semantic-Based Spatial Graph Convolution Network for Skeleton-Based Human Action Recognition (AAAI 2024 · similarity 0.90)
- Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · similarity 0.89)
- GCNext: Towards the Unity of Graph Convolutions for Human Motion Prediction (AAAI 2024 · similarity 0.88)
- BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition (CVPR 2024 · similarity 0.88)
- Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.