Research map / Video / Action recognition
Fully supervised temporal action detection
Detects action boundaries using temporal convolution, multi-task recycling and context aggregation.
87 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Action recognition, keyword cluster “temporal action · action localization · action detection”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
15 papers in 2023–24 and 17 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 10.6% to 12.9%.
No longer reported by papers here: GTEA (accuracy, last in 2021); THUMOS14 (average mAP, last in 2024); Breakfast (Acc, last in 2022).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · End-To-End Learning of Action Detection From Frame Glimpses in Videos (CVPR 2016 · 627 citations)
- 2017 · Temporal Convolutional Networks for Action Segmentation and Detection (CVPR 2017 · 2,175 citations)
- 2018 · AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions (CVPR 2018 · 1,021 citations)
- 2019 · MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation (CVPR 2019 · 685 citations)
- 2020 · G-TAD: Sub-Graph Localization for Temporal Action Detection (CVPR 2020 · 489 citations)
- 2021 · Temporal Context Aggregation Network for Temporal Action Proposal Refinement (CVPR 2021 · 183 citations)
- 2022 · MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detection (CVPR 2022 · 108 citations)
- 2023 · TriDet: Temporal Action Detection With Relative Boundary Modeling (CVPR 2023 · 213 citations)
- 2024 · End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames (CVPR 2024 · 53 citations)
- 2025 · Context-Enhanced Memory-Refined Transformer for Online Action Detection (CVPR 2025 · 10 citations)
- 2026 · FineTec: Fine-Grained Action Recognition Under Temporal Corruption via Skeleton Decomposition and Sequence Completion (AAAI 2026 · 2 citations)
Earliest papers here
- End-To-End Learning of Action Detection From Frame Glimpses in Videos (CVPR 2016 · 627 citations)
- A Multi-Stream Bi-Directional Recurrent Neural Network for Fine-Grained Action Detection (CVPR 2016 · 473 citations)
- Fast Temporal Activity Proposals for Efficient Detection of Human Actions in Untrimmed Videos (CVPR 2016 · 304 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
GTEA · accuracy (higher is better)
5 papers, 2017–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2017 | 64 | Temporal Convolutional Networks for Action Segmentation and Detection (CVPR 2017 · 2,175 citations) |
| 2018 | 70.1 | Temporal Deformable Residual Networks for Action Segmentation in Videos (CVPR 2018 · 206 citations) |
| 2019 | 79.2 | MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation (CVPR 2019 · 685 citations) |
THUMOS14 · average mAP (higher is better)
5 papers, 2022–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 54.2 | An Empirical Study of End-to-End Temporal Action Detection (CVPR 2022 · 76 citations) |
| 2023 | 69.3 | TriDet: Temporal Action Detection With Relative Boundary Modeling (CVPR 2023 · 213 citations) |
| 2024 | 76.9 | End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames (CVPR 2024 · 53 citations) |
Breakfast · Acc (higher is better)
4 papers, 2019–2022 · not reported since 2022
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 66.3 | MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation (CVPR 2019 · 685 citations) |
| 2021 | 70.7 | Global2Local: Efficient Structure Search for Video Action Segmentation (CVPR 2021 · 74 citations) |
| 2022 | 75.8 | Don't Pour Cereal into Coffee: Differentiable Temporal Logic for Temporal Action Segmentation (NeurIPS 2022 · 6 citations) |
THUMOS’14 · mAP (higher is better)
4 papers, 2022–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2022 | 72.5 | GateHUB: Gated History Unit With Background Suppression for Online Action Detection (CVPR 2022 · 64 citations) |
| 2025 | 73.2 | Context-Enhanced Memory-Refined Transformer for Online Action Detection (CVPR 2025 · 10 citations) |
What has been tried
73 papers here report 834 measured changes (a component added, removed or swapped, compared with a baseline): helped 75%, no clear effect 5%, hurt 17%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase input from 768 to 3072 frames | THUMOS14 · average mAP | 68.8 -> 70.6 | Helped | End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames (CVPR 2024 · 53 citations) |
| Use time-aligned coordinate expression instead of normalized coordinates | THUMOS14 · mAP@AVG | 40.2 -> 59.5 | Helped | TE-TAD: Towards Full End-to-End Temporal Action Detection via Time-Aligned Coordinate Expression (CVPR 2024 · 22 citations) |
| add 4 global inner-backbone propagation blocks | THUMOS14 validation · average mAP | 59.4 -> 62.3 | Helped | Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · 20 citations) |
| Add FrameDrop only | THUMOS14-C · corrupted mAP | 51.71 -> 52.56 | Helped | Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions (CVPR 2024 · 11 citations) |
| add near-future generation and memory refinement | THUMOS’14 · mAP | 71.5 -> 73.0 | Helped | Context-Enhanced Memory-Refined Transformer for Online Action Detection (CVPR 2025 · 10 citations) |
| Add GRU | 50Salads · Edit | 19.6 to 27.1 | Helped | OnlineTAS: An Online Baseline for Temporal Action Segmentation (NeurIPS 2024 · 7 citations) |
| add prediction-feedback objectives (SA encoder, SA decoder, CA decoder) to DETR | ActivityNet-v1.3 · Avg. mAP (0.5-0.95) | 34.14 -> 38.62 | Helped | Prediction-Feedback DETR for Temporal Action Detection (AAAI 2025 · 6 citations) |
| increase AAPL sampling interval from 10 to 30 seconds | ActivityNet 1.3 · Avg mAP | 25.7 -> 24.7 | Hurt | Action-Agnostic Point-Level Supervision for Temporal Action Detection (AAAI 2025 · 4 citations) |
| Jointly train segmentation and anticipation, including segmentation loss on observed frames | 50 Salads LTA (alpha=0.2, beta=0.1) · MoC | 27.32 -> 39.55 | Helped | ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation (NeurIPS 2024 · 4 citations) |
| Add current frame-text contrastive loss | THUMOS’14 · mAP | 32.9 -> 36.3 | Helped | Does Video-Text Pretraining Help Open-Vocabulary Online Action Detection? (NeurIPS 2024 · 3 citations) |
What reviewers will expect
Datasets and baselines named in the 6 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- THUMOS'14 · 4 of 6 papers
- FineAction · 2 of 6 papers
- ActivityNet-v1.3 · 2 of 6 papers
Baselines
- ActionFormer · 2 of 6 papers
- TriDet · 2 of 6 papers
- DyFaDet · 2 of 6 papers
- TadTR · 2 of 6 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 32 papers; from OpenAlex and the papers' affiliations).
- Beihang University · 2 papers
- Nanjing University · 2 papers
- Shanghai AI Lab · 2 papers
- National University of Singapore · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Temporal action segmentation (88 papers)
- Weakly supervised temporal action localization (79 papers)
- Action anticipation and latent actions (79 papers)
Elsewhere in the topic
- Disentangled action recognition (182 papers)
- Spatiotemporal CNNs for action recognition (82 papers)
- Contrastive video representation learning (74 papers)
- Skeleton-based action recognition (46 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).