Research map / Video / Video-language understanding
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
Yi Wang, Kunchang Li, Xinhao Li, Jiashuo Yu, Yinan He, Guo Chen, Baoqi Pei, Rongkun Zheng, Jilan Xu, Zun Wang, Yansong Shi, Tianxiang Jiang and 6 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale video encoder from 1B to 6B (zero-shot action recognition average) (ablation) | InternVideo2-s2 · zero-shot action recognition average | accuracy (%): 55.5 -> 56.9 | Improved |
| scale video encoder from 1B to 6B (finetuned action recognition) (ablation) | InternVideo2-s1 · finetuned action recognition average | accuracy (%): 73.2 -> 73.6 | No clear effect |
| add multimodal stage 2 pretraining before linear probing (ablation) | InternVideo2-6B · Something-Something V2 linear probing | top-1 accuracy (%): 47.8 -> 56.7 | Improved |
| use AutoShot semantic segmentation instead of SceneDet filter (ablation) | InternVideo2 stage 2 data · MSR-VTT zero-shot T2V | R@1: 27.1 -> 34.8 | Improved |
| add audio captions to video captions (ablation) | InternVideo2 stage 2 data · MSR-VTT zero-shot T2V | R@1: 24.7 -> 26.6 | Improved |
| add speech captions to video and audio captions (ablation) | InternVideo2 stage 2 data · MSR-VTT zero-shot T2V | R@1: 26.6 -> 27.1 | Improved |
| add audio encoder trained with video-audio-text learning (ablation) | stage 2 baseline · MSR-VTT zero-shot T2V (ViT-B) | R@1: 24.7 -> 27.8 | Improved |
| add both audio and speech encoders (ablation) | stage 2 baseline · MSR-VTT zero-shot T2V (ViT-B) | R@1: 27.8 -> 25.7 | Mixed |
| distill from both multimodal and motion-aware teachers instead of multimodal teacher only (ablation) | stage 1 ViT-1B · SSv2 finetuned | top-1 accuracy (%): 76.5 -> 77.1 | Improved |
| enlarge pretraining data from K710 to K-Mash 1.1M (ablation) | stage 1 ViT-1B · finetuned action recognition average | accuracy (%): 78.5 -> 79.0 | Improved |
| inject questions into the QFormer during stage 3 instruction tuning (ablation) | VideoChat2 with InternVideo2 · Egoschema-full | accuracy (%): 55.8 -> 52.9 | Hurt |
| scale ViT from 1B to 6B for fine-grained localization (ablation) | InternVideo2-s1 · FineAction TAL | avg mAP: 27.2 -> 27.7 | Mixed |
Closest papers by meaning
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation (ICLR 2024 · similarity 0.92)
- MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos (NeurIPS 2026 · similarity 0.89)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.89)
- MVBench: A Comprehensive Multi-modal Video Understanding Benchmark (CVPR 2024 · similarity 0.88)
- Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (ICLR 2025 · similarity 0.88)
- HourVideo: 1-Hour Video-Language Understanding (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
