AI Paper Atlas

Research map / Video / Action recognition

Contrastive Learning of Global and Local Video Representations

Shuang Ma, Zhaoyang Zeng, Daniel McDuff, Yale Song

NeurIPS 2021 · Video › Action recognition · approach: Action anticipation and latent actions · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the spatially-global/temporally-local objective L^l to L^g (ablation)global-local audio-visual contrastive model · LRS2WER: 70.9 -> 47.6Improved
add the L^g objective to L^l (ablation)global-local audio-visual contrastive model · LRWtop-1 accuracy: 65.1 -> 88.9Improved
use two visual encoders (slow and fast) instead of only E_v^l (ablation)global-local model · LRWtop-1 accuracy: 81.2 -> 88.9Improved
use features from both subspaces downstream instead of z_v^l only (ablation)global-local model · HMDB51accuracy: 58.3 -> 61.3Improved
use spatial attention pooling from audio-visual map instead of full-frame features (ablation)global-local model · DFDCAUC: 68.1 -> 95.9Improved
use lip/face crop instead of attention pooling on full frame (ablation)global-local model · LRWtop-1 accuracy: 85.1 -> 89.2Improved
sum over multiple positive pairs with no pooling instead of average pooling single pair in L^l (ablation)L^l objective · LRS2/LRW/DFDCWER/accuracy/AUC: LRW 79.2 -> 83.7, DFDC 88.9 -> 90.1, LRS2 40.4 -> 47.8Mixed
audio window M=1 vs M in {3,5} (ablation)L^l objective · downstream tasksperformanceHurt
use global-local contrastive objective instead of MoCovideo pretraining · LRWtop-1 accuracy: 61.2 -> 83.7Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy