Research map / Video / Video-language understanding
Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a generated caption as a training positive pair (ablation) | CLIP text-video retrieval · MSR-VTT 1K validation, global matching | R@1: 42.8 -> 44.2 | Helped |
| add a generated caption as a training positive pair (ablation) | CLIP text-video retrieval · MSR-VTT 1K validation, fine-grained matching | R@1: 45.7 -> 46.3 | Helped |
| use three or five filtered captions rather than one (ablation) | Cap4Video data augmentation · MSR-VTT 1K validation, global matching | R@1: 44.2 -> 43.3; 44.2 -> 43.4 | Hurt |
| add video-caption feature interaction by summation (ablation) | Cap4Video fine-grained matching · MSR-VTT 1K validation | R@1: 46.3 -> 47.2 | Helped |
| add video-caption feature interaction by summation (ablation) | Cap4Video global matching · MSR-VTT 1K validation | R@1: 44.2 -> 43.8 | Hurt |
| use concatenation and an MLP for video-caption interaction (ablation) | Cap4Video · MSR-VTT 1K validation, global and fine-grained matching | R@1: 44.2 -> 37.5; 46.3 -> 40.0 | Hurt |
| add video-caption interaction with a Cross Transformer (ablation) | Cap4Video · MSR-VTT 1K validation, global and fine-grained matching | R@1: 44.2 -> 44.6; 46.3 -> 47.9 | Helped |
| add video-caption interaction with a Co-attention Transformer (ablation) | Cap4Video · MSR-VTT 1K validation, global and fine-grained matching | R@1: 44.2 -> 45.3; 46.3 -> 48.5 | Helped |
| add query-caption matching scores to query-video matching scores (ablation) | Cap4Video fine-grained matching · MSR-VTT 1K validation | R@1: 48.5 -> 49.3 | Helped |
| replace query-video matching with query-caption matching alone (ablation) | Cap4Video · MSR-VTT 1K validation, fine-grained matching | R@1: 48.5 -> 30.3 | Hurt |
| use a generated CLIP+GPT-2 caption instead of a video-only baseline (ablation) | Cap4Video global matching · MSR-VTT 1K-A | R@1: 42.8 -> 45.6 | Helped |
Closest papers by meaning
- Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captions (CVPR 2025 · similarity 0.89)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.89)
- Exploiting Auxiliary Caption for Video Grounding (AAAI 2024 · similarity 0.88)
- DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval (CVPR 2025 · similarity 0.87)
- Holistic Features are almost Sufficient for Text-to-Video Retrieval (CVPR 2024 · similarity 0.87)
- Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.