Research map / Graph learning / Knowledge and temporal graphs
BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BlockGCN, which adds static (shortest-path) and dynamic (persistent homology) topological encodings to GCNs and a block-diagonal BlockGC layer to cut parameters while improving skeleton-based action recognition.
Problem
Learnable adjacency matrices in GCNs for skeleton-based action recognition gradually forget physical bone connectivity during training, and existing multi-relational modeling (ensembles, attention) is parameter-heavy and redundant.
Approach
Encode bone connectivity as a trainable table indexed by shortest-path distance between joints and add an input-dependent persistent-homology descriptor, while replacing the dense feature projection with a block-diagonal weight matrix (BlockGC) so that feature groups are modeled independently with fewer parameters.
Key results
- BlockGCN with 4 modalities reaches 93.1% on NTU RGB+D X-Sub and 90.3% on NTU RGB+D 120 X-Sub with 1.3M parameters and 1.63G FLOPs.
- BlockGC cuts spatial projection parameters by about 43% (2.1M to 1.2M) and improves ablation accuracy from 85.2% to 85.8% on NTU RGB+D 120 X-Sub (joint only).
- With joint modality alone, BlockGCN reaches 86.9% on NTU RGB+D 120 X-Sub in ablation, and 90.9% on NTU RGB+D 60 X-Sub.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BlockGCN instead of FR Head | Skeleton-based action recognition · NTU RGB+D 120 X-Sub | accuracy: 89.5 -> 90.3 | Helped |
| Use BlockGCN instead of FR Head | Skeleton-based action recognition · NTU RGB+D 120 X-Set | accuracy: 90.9 -> 91.5 | Helped |
| Replace vanilla GC with BlockGC (ablation) | GCN on NTU RGB+D 120 X-Sub · NTU RGB+D 120 X-Sub | accuracy: 85.2 -> 85.8 | Helped |
| Add learnable absolute positional embedding (ablation) | BlockGC model · NTU RGB+D 120 X-Sub | accuracy: 85.8 -> 86.0 | Helped |
| Add static topological encoding (ablation) | BlockGC model with PE · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 86.2 | Helped |
| Add dynamic topological encoding (ablation) | BlockGC model with PE · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 86.7 | Helped |
| Add both static and dynamic topological encodings (ablation) | BlockGC model with PE · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 86.9 | Helped |
| Use feature-wise instead of shared dynamic encoding (ablation) | Dynamic topological encoding · NTU RGB+D 120 X-Sub | accuracy: 86.5 -> 86.9 | Helped |
| Use feature-wise instead of shared static encoding (ablation) | Static topological encoding · NTU RGB+D 120 X-Sub | accuracy: 86.9 -> 86.7 | Hurt |
| Use BlockGC instead of DecouplingGC (ablation) | GCN on NTU RGB+D 120 X-Sub · NTU RGB+D 120 X-Sub | accuracy: 85.6 -> 85.8 | Helped |
| Increase the number of encoding layers from 0 to 10 (ablation) | BlockGC model · NTU RGB+D 120 X-Sub | accuracy: 86.0 -> 86.9 | Helped |
| Increase barcode vectorization feature size from 32 to 64 (ablation) | Dynamic topological encoding · NTU RGB+D 120 X-Sub | accuracy: 86.5 -> 86.9 | Helped |
Surprising findings
- Learned adjacency matrices of GCNs deviate strongly from the bone connections used for initialization, and GCN accuracy is similar regardless of initialization.
- Ensemble-of-GCs redundancy: a large portion of the convolution weight matrix is redundant, and BlockGC with 8 groups gives higher accuracy with fewer parameters than the dense baseline.
Limitations
- Strongest reported numbers rely on 4-stream (or multi-modality) fusion; single-joint gains are smaller.
- Persistent homology details are deferred to supplementary material, so the dynamic encoding is harder to reproduce from the main text.
- Comparisons with methods using extra supervision or extra modalities (GAP, HDGCN, PoseC3D) are not direct.
Experimental setup
Datasets: NTU RGB+D NTU RGB+D 120 Northwestern-UCLA
Benchmarks: NTU RGB+D NTU RGB+D 120 Northwestern-UCLA
Metrics: top-1 accuracy (X-Sub) top-1 accuracy (X-View) top-1 accuracy (X-Set) parameters FLOPs
Baselines: DecouplingGCN MS-G3D MST-GCN CTR-GCN EfficientGCN-B4 InfoGCN FR Head HDGCN CTR-GCN* (GAP) SkeletonGCL
Builds on: ST-GCN CTR-GCN InfoGCN DecouplingGCN MS-G3D
Closest papers by meaning
- Topology-Aware Convolutional Neural Network for Efficient Skeleton-Based Action Recognition (AAAI 2022 · similarity 0.94)
- Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.92)
- Dynamic Semantic-Based Spatial Graph Convolution Network for Skeleton-Based Human Action Recognition (AAAI 2024 · similarity 0.92)
- Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition (CVPR 2019 · similarity 0.92)
- Actional-Structural Graph Convolutional Networks for Skeleton-Based Action Recognition (CVPR 2019 · similarity 0.92)
- Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.