Research map / Video / Action recognition
Skeleton-based action recognition
Recognizes actions from skeleton sequences with graph convolutions, LSTMs and prototypes.
46 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Action recognition, keyword cluster “action recognition · video recognition · representation”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports None of the tracked benchmarks has a report from 2025–26.
11 papers in 2023–24 and 11 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 7.8% to 8.3%.
No longer reported by papers here: NTU RGB+D Cross-Subject (accuracy, last in 2019); NTU RGB+D Cross-View (accuracy, last in 2019).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (CVPR 2016 · 3,062 citations)
- 2017 · An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data (AAAI 2017 · 862 citations)
- 2018 · Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition (CVPR 2018 · 466 citations)
- 2019 · An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition (CVPR 2019 · 945 citations)
- 2020 · Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition (CVPR 2020 · 655 citations)
- 2021 · Spatio-Temporal Difference Descriptor for Skeleton-Based Action Recognition (AAAI 2021 · 8 citations)
- 2022 · Towards To-a-T Spatio-Temporal Focus for Skeleton-Based Action Recognition (AAAI 2022 · 73 citations)
- 2023 · Learning Discriminative Representations for Skeleton Based Action Recognition (CVPR 2023 · 208 citations)
- 2024 · Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · 19 citations)
- 2025 · Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · 36 citations)
- 2026 · Gait Transformer: End-to-End Transformer Backbone for Gait Recognition (AAAI 2026 · 3 citations)
Earliest papers here
- NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (CVPR 2016 · 3,062 citations)
- Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks (AAAI 2016 · 902 citations)
- Regularizing Long Short Term Memory With 3D Human-Skeleton Sequences for Action Recognition (CVPR 2016 · 181 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
NTU RGB+D Cross-Subject · accuracy (higher is better)
4 papers, 2016–2019 · not reported since 2019
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 62.93 | NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (CVPR 2016 · 3,062 citations) |
| 2017 | 79.57 | A New Representation of Skeleton Sequences for 3D Action Recognition (CVPR 2017 · 789 citations) |
| 2019 | 89.2 | An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition (CVPR 2019 · 945 citations) |
NTU RGB+D Cross-View · accuracy (higher is better)
4 papers, 2016–2019 · not reported since 2019
| Year | Best so far | Set by |
|---|---|---|
| 2016 | 70.27 | NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis (CVPR 2016 · 3,062 citations) |
| 2017 | 84.83 | A New Representation of Skeleton Sequences for 3D Action Recognition (CVPR 2017 · 789 citations) |
| 2019 | 95 | An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition (CVPR 2019 · 945 citations) |
What has been tried
37 papers here report 385 measured changes (a component added, removed or swapped, compared with a baseline): helped 74%, no clear effect 5%, hurt 19%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add Motion Topology Enhancement (MTE) | NTU RGB+D 120 X-Sub (bone modality) · accuracy | 87.8 -> 88.1 | Helped | Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition (CVPR 2025 · 36 citations) |
| Fuse JTG frame counts 2, 4, 6, and 8 rather than using N=1 | NTU-60 X-Sub · Top-1 accuracy (%) | 87.4 -> 91.7 | Helped | Spatio-Temporal Fusion for Human Action Recognition via Joint Trajectory Graph (AAAI 2024 · 19 citations) |
| add pretraining | Tactile action classification test set · ACC@1 | 0.8050 -> 0.9033 | Helped | Jointly Modeling Spatio-Temporal Features of Tactile Signals for Action Classification (AAAI 2024 · 3 citations) |
| add skeleton constraint F | Diving48 · Acc@1 | 68.17 -> 72.99 | Helped | H-MoRe: Learning Human-centric Motion Representation for Action Analysis (CVPR 2025 · 2 citations) |
| apply linear transforms only | Cross-dataset average (four settings) · balanced accuracy | 54.4 -> 57.2 | Helped | Recovering Complete Actions for Cross-dataset Skeleton Action Recognition (NeurIPS 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 5 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- NTU RGB+D · 2 of 5 papers
- NTU RGB+D 120 · 2 of 5 papers
Baselines
- ST-GCN · 2 of 5 papers
- 2s-AGCN · 2 of 5 papers
- MS-G3D · 2 of 5 papers
- CTR-GCN · 2 of 5 papers
- InfoGCN · 2 of 5 papers
- SkeletonMAE · 2 of 5 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 22 papers; from OpenAlex and the papers' affiliations).
- Peng Cheng Laboratory · 2 papers
- Peking University · 2 papers
- Tsinghua University · 2 papers
Use it
Download BibTeX of 34 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Disentangled action recognition (182 papers)
- Spatiotemporal CNNs for action recognition (82 papers)
- Contrastive video representation learning (74 papers)
Elsewhere in the topic
- Temporal action segmentation (88 papers)
- Fully supervised temporal action detection (87 papers)
- Weakly supervised temporal action localization (79 papers)
- Action anticipation and latent actions (79 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).