AI Paper Atlas

Research map / Video / Video-language understanding

Verbs in Action: Improving Verb Understanding in Video-Language Models

Liliane Momeni, Mathilde Caron, Arsha Nagrani, Andrew Zisserman, Cordelia Schmid

ICCV 2023 · Video › Video-language understanding · approach: Video-language pretraining · 73 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add LLM-generated verb-only hard negative captions with calibration (ablation)CLIP video model on SMiT · MSR-VTT MC Verb_Haccuracy: 69.9 -> 78.0Improved
generate negatives with PaLM instead of random or antonym verbs (ablation)hard negative generation · MSR-VTT MC Verb_Haccuracy: 73.6 (random) -> 78.0 (PaLM)Improved
generate negatives with T5 instead of PaLM (ablation)hard negative generation · MSR-VTT MC Verb_Haccuracy: 78.0 (PaLM) -> 75.1 (T5)Mixed
remove calibration of hard negatives (ablation)VFC hard negative loss · Kinetics-400top-1 accuracy: 55.8 -> 54.5Hurt
remove calibration of hard negatives (ablation)VFC hard negative loss · MSR-VTT MC Verb_Haccuracy: 78.0 -> 80.5Improved
add verb phrase loss to hard negatives (ablation)VFC · Kinetics-400top-1 accuracy: 55.8 -> 58.5Improved
add verb phrase loss to hard negatives (ablation)VFC · MSR-VTT MC Verb_Haccuracy: 78.0 -> 76.3Hurt
increase max hard negative captions from 1 to 5 (ablation)VFC · MSR-VTT MC Verb_Haccuracy: 76.3 -> 78.3Improved
use HardNeg-NCE instead of standard NCE (ablation)VFC · MSR-VTT MC Verb_Haccuracy: 78.3 -> 80.5Improved
mine matching videos for generated captions instead of calibration (ablation)VFC hard negatives · Kinetics-400top-1 accuracy: 55.8 -> 51.8Hurt
generate positive synonym-verb captions instead of negatives (ablation)VFC · verb benchmarksaccuracyHurt
add input-output example pairs to PaLM prompt (ablation)hard negative generation · MSR-VTT MC Verb_Haccuracy: 77.5 -> 78.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy