AI Paper Atlas

Research map / Video / Video-language understanding

Learning Text-to-Video Retrieval from Image Captioning

Lucas Ventura, Cordelia Schmid, Gül Varol

IJCV 2024 · Video › Video-language understanding · approach: Text-video retrieval · 13 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
finetune CLIP with BLIP-captioned pseudo labelsCLIP baseline · MSR-VTTT2V R@1: 32.8 -> 35.8Improved
add CLIPScore top-2 selection instead of random of 10 captions (ablation)ClipCap labels · MSR-VTTT2V R@1: 31.8 -> 34.7Improved
use top-1 caption instead of middle-frame caption (ablation)BLIP labels · ActivityNetT2V R@1: 25.7 -> 27.6Improved
increase selected captions from top 2 to top 3 (ablation)ClipCap labels · MSR-VTTT2V R@1: 34.7 -> 33.1Hurt
add BLIP captions to ClipCap captions (ablation)ClipCap · MSR-VTTT2V R@1: 34.7 -> 36.5Improved
use multi-caption query-scoring instead of single random caption (ablation)random 4 captions · MSR-VTTT2V R@1: 37.6 -> 39.0Improved
concatenate the 4 captions into one text (ablation)MCQS · MSR-VTTT2V R@1: 39.0 -> 27.3Hurt
weight caption similarities by CLIPScore instead of mean (ablation)MCQS · MSR-VTTT2V R@1: 39.0 -> 38.6Hurt
add query-scoring at evaluation to CLIP baseline (ablation)CLIP baseline · MSR-VTTT2V R@1: 32.8 -> 33.9Improved
train on three datasets combined instead of one (ablation)Ours · MSVDT2V R@1: 42.5 -> 44.6Improved
train on three datasets combined instead of one (ablation)Ours · ActivityNetT2V R@1: 29.7 -> 30.6Improved
add OFA as third captioner (ablation)MCQS with ClipCap+BLIP · MSVDT2V R@1 / R@5: R@1 42.5 -> 42.8, R@5 70.0 -> 68.5Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy