AI Paper Atlas

Research map / Video / Action recognition

PALM: Predicting Actions through Language Models

Sanghwan Kim, Daoji Huang, Yongqin Xian, Otmar Hilliges, Luc Van Gool, Xi Wang

ECCV 2024 · Video › Action recognition · approach: Action anticipation and latent actions · 32 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use MMR exemplar selection instead of random selection (ablation)PALM · Ego4D v2 valAction ED: 0.9002 -> 0.8908Improved
use MMR instead of similarity-only selection (ablation)PALM · Ego4D v2 valAction ED: 0.8923 -> 0.8908Improved
add image captions to the action-sequence prompt (ablation)PALM · Ego4D v2 valAction ED: 0.8947 -> 0.8908Improved
add past action sequence to the captions-only prompt (ablation)PALM · Ego4D v2 valAction ED: 0.9064 -> 0.8908Improved
use a larger LLM (Llama 2 13B instead of 7B) (ablation)PALM · Ego4D v2 valAction ED: 0.8908 -> 0.8867Improved
use EgoVLP video encoder instead of SlowFast for action recognition (ablation)PALM · Ego4D v2 valAction ED: 0.9278 -> 0.8819Improved
use ground truth past actions instead of recognized actions (ablation)PALM · Ego4D v2 valAction ED: 0.8819 -> 0.8319Improved
increase in-context exemplars from 1 to 4 (ablation)PALM · Ego4D v2 valAction ED: 0.8908 -> 0.8859Improved
use 4 input segments instead of 1 for action recognition (ablation)action recognition module · Ego4D v2 valaction accuracy: 17.02 -> 20.63Improved
use 8 input segments instead of 4 (ablation)action recognition module · Ego4D v2 valaction accuracy: 20.63 -> 17.57Hurt
use classification heads instead of video-text retrieval for action recognition (ablation)action recognition module · Ego4D v2 valaction accuracy: 15.18 -> 20.63Improved
swap the captioning model (BLIP to VideoBLIP) (ablation)PALM · Ego4D v2 valAction ED: 0.8927 -> 0.8907No clear effect

Later papers that compared with this method

4 later accepted papers report PALM as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy