Research map / Video / Video-language understanding
Learning Text-to-Video Retrieval from Image Captioning
Lucas Ventura, Cordelia Schmid, Gül Varol
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune CLIP with BLIP-captioned pseudo labels | CLIP baseline · MSR-VTT | T2V R@1: 32.8 -> 35.8 | Improved |
| add CLIPScore top-2 selection instead of random of 10 captions (ablation) | ClipCap labels · MSR-VTT | T2V R@1: 31.8 -> 34.7 | Improved |
| use top-1 caption instead of middle-frame caption (ablation) | BLIP labels · ActivityNet | T2V R@1: 25.7 -> 27.6 | Improved |
| increase selected captions from top 2 to top 3 (ablation) | ClipCap labels · MSR-VTT | T2V R@1: 34.7 -> 33.1 | Hurt |
| add BLIP captions to ClipCap captions (ablation) | ClipCap · MSR-VTT | T2V R@1: 34.7 -> 36.5 | Improved |
| use multi-caption query-scoring instead of single random caption (ablation) | random 4 captions · MSR-VTT | T2V R@1: 37.6 -> 39.0 | Improved |
| concatenate the 4 captions into one text (ablation) | MCQS · MSR-VTT | T2V R@1: 39.0 -> 27.3 | Hurt |
| weight caption similarities by CLIPScore instead of mean (ablation) | MCQS · MSR-VTT | T2V R@1: 39.0 -> 38.6 | Hurt |
| add query-scoring at evaluation to CLIP baseline (ablation) | CLIP baseline · MSR-VTT | T2V R@1: 32.8 -> 33.9 | Improved |
| train on three datasets combined instead of one (ablation) | Ours · MSVD | T2V R@1: 42.5 -> 44.6 | Improved |
| train on three datasets combined instead of one (ablation) | Ours · ActivityNet | T2V R@1: 29.7 -> 30.6 | Improved |
| add OFA as third captioner (ablation) | MCQS with ClipCap+BLIP · MSVD | T2V R@1 / R@5: R@1 42.5 -> 42.8, R@5 70.0 -> 68.5 | Mixed |
Closest papers by meaning
- Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captions (CVPR 2025 · similarity 0.89)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.89)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.88)
- Holistic Features are almost Sufficient for Text-to-Video Retrieval (CVPR 2024 · similarity 0.88)
- Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning (AAAI 2024 · similarity 0.88)
- DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
