Research map / Video / Video-language understanding
Learning Beyond Still Frames: Scaling Vision-Language Models with Video
Yiyuan Zhang, Handong Li, Jing Liu, Xiangyu Yue
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale video data in Stage I pretraining (ablation) | LLaVA-Prime · VideoMME / Egoschema | accuracy: 55.1 -> 57.3 (VideoMME) | Improved |
| scale pretraining video data from 2M to 128M (ablation) | LLaVA-Prime-7B · VideoMME general | score (%): 56.2 -> 63.8 | Improved |
| scale pretraining video data from 2M to 128M (ablation) | LLaVA-Prime-7B · general VL tasks (MME, MMBench, POPE, SEED, MMMU) | score (%): 66.2 -> 72.2 | Improved |
| scale video data for long videos (ablation) | LLaVA-Prime-7B · VideoMME long | score (%): 46.9 at 16M -> 55.5 at 128M | Mixed |
| increase video frames rate to 5 fps (ablation) | LLaVA-Prime · VideoMME | accuracy: 57.5 -> 59.8 (1 fps to 5 fps) | Improved |
| increase video resolution from 224 to 336 (ablation) | LLaVA-Prime · VideoMME | accuracy: 55.1 -> 54.8 | Hurt |
| add video pretraining of LLM (Stage II) | Qwen-2-7B · CMMLU / MMLU-Pro / MMLU | accuracy (%): CMMLU +3.4, MMLU-Pro +1.0, MMLU 68.2 -> 48.6 | Mixed |
| add Stage III causal hierarchical aggregation (ablation) | LLaVA-Prime · VideoMME | accuracy: 58.8 -> 60.2 | Improved |
| scale LLM size (ablation) | LLaVA-Prime · VideoMME | accuracy: 55.1 -> 62.8 | Improved |
| video-pretrain ViT-g vision encoder vs EVA-01-CLIP-g/14 | LLaVA-Prime-ViT-g · zero-shot classification (27 datasets) | avg top-1 accuracy (%): 72.4 -> 78.5 | Improved |
Closest papers by meaning
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.91)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.91)
- Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization (ICML 2024 · similarity 0.90)
- MVBench: A Comprehensive Multi-modal Video Understanding Benchmark (CVPR 2024 · similarity 0.90)
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.90)
- Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
