AI Paper Atlas

Research map / Video / Action recognition

Action anticipation and latent actions

Predicts future actions with LLMs, graph growth and latent action world models.

79 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Action recognition, keyword cluster “temporal action · action localization · action detection”

State History Tried Checklist Preprints Who Export

Where it stands

Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.

29 papers in 2023–24 and 29 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 20.6% to 22.0%.

How this is decided

A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.

How it got here

2016: 0162017: 1172018: 4182019: 3192020: 0202021: 7212022: 6222023: 14232024: 15242025: 12252026: 1726

Most cited paper of each year

Earliest papers here

What has been tried

58 papers here report 628 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 4%, hurt 19%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.

Ablations in recent papers

What was changedMeasured onResultEffectPaper
Add gaze-cropped ego video during zero-shot InternVideo transferCross-view association test, Ego2Exo · Top-1 accuracy27.0 -> 30.9HelpedEgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World (CVPR 2024 · 40 citations)
add a second dynamic prior instead of using onenuScenes · Average Trajectory Difference3.785 -> 2.597HelpedVista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability (NeurIPS 2024 · 38 citations)
Add the plausible action sequence learning loss (Lplau)Ego4D · ED@(Z=20) verb0.691 -> 0.679HelpedCan't Make an Omelette Without Breaking Some Eggs: Plausible Action Anticipation Using Large Video-Language Models (CVPR 2024 · 26 citations)
Represent actions with random symbols rather than numerical labelsAssembly101-O · F1 score39.9 -> 41.8HelpedPREGO: Online Mistake Detection in PRocedural EGOcentric Videos (CVPR 2024 · 18 citations)
Use tokenized images rather than pixels as latent action model inputPlatformers · PSNRdiff1.91 -> 1.33HurtGenie: Generative Interactive Environments (ICML 2024 · 12 citations)
Add label augmentationPangea test set · mAP34.01 -> 34.25HelpedFrom Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding (CVPR 2024 · 10 citations)
Use a 3-layer encoder and 4-layer future decoderEK100 validation · verb Top-1 accuracy34.5 -> 36.9HelpedTowards Understanding Future: Consistency Guided Probabilistic Modeling for Action Anticipation (AAAI 2024 · 8 citations)
Use GPT-3-curie as the temporal LLM and add inferred goalsEgo4D v1 validation · Verb edit distance0.707 -> 0.709HurtAntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos? (ICLR 2024 · 8 citations)
Single unified TGT-text model across all procedures vs per-procedure trainingCaptainCook4D graph generation · F180.8 to 64.5HurtDifferentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos (NeurIPS 2024 · 7 citations)
Use task-related graph vertices rather than grid-sampled vertices or point-based flowRobot-to-robot transfer, Libero-Spatial and Libero-Goal · success rate0.51 -> 0.81 (Libero-Spatial); 0.53 -> 0.81 (Libero-Goal)HelpedGraphMimic: Graph-to-Graphs Generative Modeling from Videos for Policy Learning (CVPR 2025 · 4 citations)
Use error-category-specific prompt ensembling instead of a single step promptZeroShotER · F16.7 -> 41.8HelpedCaptainCook4D: A Dataset for Understanding Errors in Procedural Activities (NeurIPS 2024 · 4 citations)
add similarity estimationDWC validation set · MAE11.30 -> 10.87HelpedCount What You Want: Exemplar Identification and Few-Shot Counting of Human Actions in the Wild (AAAI 2024 · 3 citations)

What reviewers will expect

Datasets and baselines named in the 16 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.

Datasets

  • LIBERO · 2 of 16 papers

Baselines

  • BC · 2 of 16 papers
  • LLaVA-Video · 2 of 16 papers

Recent preprints

Loading recent arXiv preprints…

    Who publishes on it

    Institutions on the most papers of this approach since 2023 (of 58 papers; from OpenAlex and the papers' affiliations).

    Use it

    Download BibTeX of 37 key papersOpen the topic in the interactive map

    Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.

    Same keyword cluster

    Elsewhere in the topic

    Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).

    Get the next scime Dispatch and a monthly field update

    Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy