Research map / Graph learning / Knowledge and temporal graphs
Dynamic Semantic-Based Spatial Graph Convolution Network for Skeleton-Based Human Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DS-GCN, a skeleton-based action recognition GCN that implicitly encodes joint-type and edge-type semantics into dynamic adaptive graphs via type-specific transform functions.
Problem
Skeleton-based human action recognition with GCNs ignores semantic types of joints and edges, and explicit type encoding (e.g., one-hot inputs) is inflexible in deep networks.
Approach
Node type-aware and edge type-aware adaptive topologies use type-specific projection functions (per body part and per part-pair edge type) inside a non-local style adaptive graph, split into three branches (node, edge, general) with branch-wise learnable weights, so semantics are encoded in every ST-GCN layer.
Key results
- DS-GCN reaches 93.1% on NTU60 cross-subject, 97.5% on NTU60 cross-view, 89.2% on NTU120 cross-subject, 91.1% on NTU120 cross-setup, and 50.6% on Kinetics-400, outperforming listed SOTA methods.
- Plugging the node/edge semantic module into CTR-GCN raises NTU60 X-Sub accuracy from 89.6 to 90.4 and into 2s-GCN from 89.5 to 90.1.
- The ablation shows DS-GCN without semantic encoding reaches 90.0 on NTU60 X-Sub versus 90.8 for the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add node- and edge-type semantic modules (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.0 -> 90.8 | Helped |
| Add node- and edge-type semantic modules (ablation) | 2s-GCN · NTU60-XSub and NTU120-XSet | accuracy: 89.5 -> 90.1; 86.0 -> 86.1 | Helped |
| Add node- and edge-type semantic modules (ablation) | CTR-GCN · NTU60-XSub and NTU120-XSet | accuracy: 89.6 -> 90.4; 86.0 -> 86.5 | Helped |
| Replace ST-GCN spatial graph convolution with DS-GCN (ablation) | ST-GCN · NTU60-XSub and NTU120-XSet | accuracy: 87.8 -> 90.8; 85.5 -> 87.2 | Helped |
| Replace 2s-GCN spatial graph convolution with DS-GCN (ablation) | 2s-GCN · NTU60-XSub and NTU120-XSet | accuracy: 89.5 -> 90.8; 86.0 -> 87.2 | Helped |
| Replace CTR-GCN spatial graph convolution with DS-GCN (ablation) | CTR-GCN · NTU60-XSub and NTU120-XSet | accuracy: 89.6 -> 90.8; 86.0 -> 87.2 | Helped |
| Use individual λ for each branch instead of sharing λ (ablation) | DS-GCN · NTU60-XSub and NTU120-XSet | accuracy: 90.1 -> 90.8; 86.8 -> 87.2 | Helped |
| Add node-type encoding (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.4 -> 90.8 | Helped |
| Add edge-type encoding (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.5 -> 90.8 | Helped |
| Use DS-GCN only in layers 1–4 (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.0 -> 90.2 | Helped |
| Use DS-GCN only in layers 5–7 (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.0 -> 90.7 | Helped |
| Use DS-GCN only in layers 8–10 (ablation) | DS-GCN · NTU60-XSub | accuracy: 90.0 -> 90.5 | Helped |
Surprising findings
- Applying semantic encoding only in the middle stage of the network (layers 5-7) gave better accuracy than applying it only in the initial or end stages, which the authors attribute to over-smoothing.
Limitations
- Gains over prior methods are modest (about 0.8% over the no-semantic backbone in ablation).
- Multi-stream fusion is needed to reach the reported SOTA numbers, so single-stream comparisons are less favorable.
Experimental setup
Datasets: NTU RGB+D 60 NTU RGB+D 120 Kinetics-400
Benchmarks: NTU RGB+D 60 cross-subject NTU RGB+D 60 cross-view NTU RGB+D 120 cross-subject NTU RGB+D 120 cross-setup Kinetics-400
Metrics: top-1 accuracy
Baselines: ST-GCN SGN AS-GCN RA-GCN 2s-GCN DGNN FGCN ShiftGCN DSTA-Net MS-G3D CTR-GCN ST-GCN++
Builds on: ST-GCN CTR-GCN 2s-AGCN ST-GCN++
Closest papers by meaning
- Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching (AAAI 2020 · similarity 0.94)
- Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition (AAAI 2018 · similarity 0.94)
- BlockGCN: Redefine Topology Awareness for Skeleton-Based Action Recognition (CVPR 2024 · similarity 0.92)
- Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · similarity 0.91)
- Are Spatial-Temporal Graph Convolution Networks for Human Action Recognition Over-Parameterized? (CVPR 2025 · similarity 0.90)
- Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.