AI Paper Atlas

Research map / Video / Video-language understanding

Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning

Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Xinshuai Dong, Zhiyuan Hu, Khoi M Le, Cong-Duy Nguyen, See Kiong Ng, Anh Tuan Luu

ECCV 2024 · Video › Video-language understanding · approach: Video-language pretraining · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add LLaVA-generated video-text data (ablation)MAMA · MSRVTT-QAaccuracy: 45.6 -> 46.4Improved
use LLaVA instead of BLIP-2 or InstructBLIP for augmentation (ablation)MAMA · DiDeMo text-to-video retrievalR@1: 58.1 / 58.5 -> 62.7Improved
summarize per-frame LLaVA captions with ChatGPT instead of the grid prompt (ablation)MAMA · DiDeMo text-to-video retrievalR@1: 62.7 -> 58.7Hurt
use MLP-parameterized weighting function instead of focal loss, SPL or L2RW (ablation)MAMA · MSRVTT-QAaccuracy: 45.0 / 45.3 / 45.9 -> 46.4Improved
use meta-learning instead of joint learning for the weighting MLP (ablation)MAMA · DiDeMo text-to-video retrievalR@1: 59.2 -> 62.7Improved
increase key frames Q up to 12 in a 3x4 grid (ablation)MAMA · MSRVTT-QAaccuracy: 45.0 -> 46.4Improved
increase key frames beyond 12 to 16 (ablation)MAMA · MSRVTT-QAaccuracy: 46.4 -> 45.7Hurt
use balanced grid (3x4) instead of skewed grid (2x6) (ablation)MAMA · MSRVTT-QAaccuracy: 45.8 -> 46.4Improved
add subtractive angular margin contrastive loss with margin around 0.2CLIP-ViP · MSRVTT, DiDeMo, ActivityNet retrievalR@1: 55.7 -> 60.0 on MSRVTTImproved
apply MAMA to VGT backbone for open-ended VideoQAVGT · MSVD-QAaccuracy: 36.4 -> 37.1Improved
apply MAMA to VIOLET instead of VGT for Causal-VidQA counterfactualMAMA · Causal-VidQA (Counterfactual)accuracy: 68.2 (VGT) vs 59.5 (VIOLET)Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy