Research map / Video / Video generation
VCapsBench: A Large-scale Fine-grained Benchmark for Video Caption Quality Evaluation
Shi-Xue Zhang, Hongfa Wang, Duojun Huang, Xin Li, Xiaobin Zhu, Xu-Cheng Yin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale Qwen2.5-VL from 7B to 72B | Qwen2.5-VL · VCapsBench (GPT-4.1 judge) | AR (all): 58.7 -> 63.1 | Improved |
| scale Qwen2.5-VL from 7B to 72B | Qwen2.5-VL · VCapsBench (GPT-4.1 judge) | CR (all): 65.8 -> 70.9 | Improved |
| upgrade open Qwen2VL-7B to Qwen2.5VL-7B | Qwen VL 7B · VCapsBench (GPT-4.1 judge) | AR (all): 47.5 -> 58.7 | Improved |
| use Gemini-2.5-Pro-Preview instead of the best open model Qwen2.5-VL-72B | caption generator · VCapsBench (GPT-4.1 judge) | AR (all): 63.1 -> 76.7 | Improved |
| use Gemini-2.5-Pro-Preview instead of GPT-4o | caption generator · VCapsBench (GPT-4.1 judge) | CR (all): 74.1 -> 85.5 | Improved |
| longer captions from stronger models | VLM captions · VCapsBench (GPT-4.1 judge) | IR (all): 10.2 (GPT-4o) vs 10.3 (Gemini-2.5-Pro-Preview) | No clear effect |
| longer captions from weaker open-source models | Qwen2VL-7B, VideoLLaMA3-7B, LLaVA-Video-7B, InternVL2.5-8B · VCapsBench (GPT-4.1 judge) | IR | Hurt |
| small models on fine-grained dimensions | LLaVA-Video-7B, Qwen2VL-7B · VCapsBench (GPT-4.1 judge) | AR (Blur, Camera Movement) | Hurt |
Closest papers by meaning
- AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark (ICLR 2025 · similarity 0.92)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.91)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.91)
- G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o (AAAI 2025 · similarity 0.90)
- T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation (CVPR 2025 · similarity 0.90)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
