Research map / Video / Video-language understanding
VaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, William Yang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weight tying of embedding and softmax (ablation) | Base captioning model · VaTeX Chinese captioning test | BLEU-4: 24.4 -> 24.9 | Improved |
| add weight tying of embedding and softmax (ablation) | Base captioning model · VaTeX English captioning test | BLEU-4: 28.1 -> 28.1 | No clear effect |
| add weight tying of embedding and softmax (ablation) | Base captioning model · VaTeX captioning | parameters: 52.5M -> 39.7M | Improved |
| share the video encoder across English and Chinese | Base captioning model · VaTeX English captioning test | CIDEr: 44.3 -> 45.1 | Improved |
| share the video encoder across English and Chinese | Base captioning model · VaTeX captioning | parameters: 39.7M -> 34.9M | Improved |
| share encoder and decoder across languages | Shared Enc · VaTeX captioning | parameters: 39.7M -> 26.3M | Improved |
| share encoder and decoder across languages | Base captioning model · VaTeX English captioning test | BLEU-4: 28.1 -> 27.9 | No clear effect |
| add average video features (ablation) | Base NMT · VaTeX English->Chinese translation | BLEU-4: 26.85 -> 26.97 | No clear effect |
| add LSTM video features without attention (ablation) | Base NMT · VaTeX English->Chinese translation | BLEU-4: 26.85 -> 27.43 | Improved |
| add temporal attention over LSTM video features (ablation) | LSTM VI w/o Attn · VaTeX English->Chinese translation | BLEU-4: 27.43 -> 29.12 | Improved |
| add temporal attention over LSTM video features (ablation) | LSTM VI w/o Attn · VaTeX Chinese->English translation | BLEU-4: 24.76 -> 26.42 | Improved |
| add video context under 100% noun masking | NMT · VaTeX Chinese->English masked translation | noun recovery accuracy: 10.1 -> 39.7 | Improved |
Closest papers by meaning
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation (ICLR 2024 · similarity 0.86)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.86)
- Semantic and Expressive Variations in Image Captions Across Languages (CVPR 2025 · similarity 0.86)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.86)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.86)
- VidLA: Video-Language Alignment at Scale (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
