Research map / Video / Video-language understanding
VTC: Improving Video-Text Retrieval with User Comments
Laura Hanu, James Thewlis, Yuki M. Asano, Christian Rupprecht
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Context Adapter Module instead of averaging comment features (ablation) | fine-tuned CLIP · VTC test (title adapted, 5 comments) | TVR R@1: 16.6 -> 18.4 | Improved |
| add comments via Context Adapter Module | fine-tuned CLIP · VTC test (title adapted, 5 comments) | TVR R@1: 15.5 -> 18.4 | Improved |
| fine-tune the CLIP encoder on VTC | CLIP ViT-B/32 · VTC test | TVR R@1: 11.1 -> 15.5 | Improved |
| average comment features with title, zero-shot | CLIP ViT-B/32 · VTC test (title adapted, 5 comments) | TVR R@1: 11.1 -> 7.3 | Hurt |
| adapt the image branch instead of the title branch (ablation) | CAM · VTC test (adaptation modality) | TVR R@1: 18.0 -> 28.2 | Improved |
| train with comments but evaluate without comments | CAM · VTC test | TVR R@10: 34.9 -> 34.5 | No clear effect |
| add audio to img+title+comments (ablation) | CAM · VTC test | TVR R@1: 18.0 -> 19.6 | Improved |
| increase comments from 5 to 20 (ablation) | CAM, CLIP ViT-B/32 · VTC test (frozen encoder + CAM, 5 comments, image from text+comments) | R@10: 32.3 -> 34.1 | Improved |
| add 5 comments | CAM, CLIP ViT-L/14 · VTC test (frozen encoder + CAM, 5 comments, image from text+comments) | R@10: 32.9 -> 42.0 | Improved |
| remove videos containing faces from training (ablation) | CAM · VTC test | TVR R@10: 43.2 -> 44.0 | No clear effect |
| add comments to 8-frame video model | CAM · KineticsComments VTR R@10 | VTR R@10: 56.9 -> 68.0 | Improved |
| adapt text branch instead of video branch for video model (ablation) | CAM · VTC and KineticsComments VTR R@10 (8 frames) | VTR R@10: 28.6 -> 33.7 (VTC), 57.5 -> 57.3 (KineticsComments) | Mixed |
Closest papers by meaning
- Multi-Event Video-Text Retrieval (ICCV 2023 · similarity 0.88)
- MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence (NeurIPS 2025 · similarity 0.86)
- MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval (CVPR 2024 · similarity 0.86)
- Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval (AAAI 2024 · similarity 0.86)
- Bridging Information Asymmetry in Text-video Retrieval: A Data-centric Approach (ICLR 2025 · similarity 0.86)
- Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
