Research map / Video / Video-language understanding
Advancing High-Resolution Video-Language Representation With Large-Scale Video Transcriptions
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train on HD-VILA-100M rather than HowTo100M for the same steps (ablation) | HD-VILA · MSR-VTT retrieval | R@1: 19.6 -> 30.0 | Helped |
| use diversified categories instead of a HowTo-only subset (ablation) | HD-VILA · MSR-VTT zero-shot retrieval | R@1: 3.3 -> 5.5 | Helped |
| use 720p source video rather than downsampled 360p video (ablation) | HD-VILA · MSR-VTT zero-shot retrieval | R@1: 3.9 -> 5.5 | Helped |
| use 720p source frames rather than bicubic-upsampled frames (ablation) | HD-VILA · MSR-VTT zero-shot retrieval | R@1: 4.5 -> 5.5 | Helped |
| combine one high-resolution frame with 10 low-resolution neighboring frames (ablation) | HD-VILA hybrid video encoder · MSR-VTT retrieval | R@1: 16.3 -> 35.6; 26.7 -> 35.6 | Helped |
| use 10 rather than 6 low-resolution frames alongside one high-resolution frame (ablation) | HD-VILA hybrid video encoder · MSR-VTT retrieval | R@1: 33.0 -> 35.6 | Helped |
| increase low-resolution frames from 10 to 14 (ablation) | HD-VILA hybrid video encoder · MSR-VTT retrieval | R@1: 35.6 -> 33.7 | Hurt |
| pre-train on in-domain HowTo100M instead of HD-VILA-100M (ablation) | HD-VILA · YouCook2 retrieval | R@1: 6.6 -> 15.7 | Helped |
| increase HD-VILA-100M pre-training from 145K to 504K steps (ablation) | HD-VILA · YouCook2 retrieval | R@1: 6.6 -> 9.1 | Helped |
| use HD-VILA rather than VideoCLIP | video-language retrieval · MSR-VTT zero-shot | R@1: 10.4 -> 14.6 | Helped |
| use HD-VILA rather than ClipBERT | video-language retrieval · DiDeMo | R@1: 20.4 -> 28.8 | Helped |
| use HD-VILA rather than MMT | video-language retrieval · LSMDC | R@1: 12.9 -> 17.4 | Helped |
Closest papers by meaning
- LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment (ICLR 2024 · similarity 0.91)
- Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization (ICML 2024 · similarity 0.90)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.89)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.89)
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation (ICLR 2024 · similarity 0.89)
- Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.