Research map / Video / Video-language understanding
VLG: General Video Recognition with Web Textual Knowledge
Jintao Lin, Zhaoyang Liu, Wenhai Wang, Wayne Wu, Limin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add video-language pretraining (VLP) (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 76.0 -> 81.8 | Improved |
| initialize with CLIP pretrained weights instead of random (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 32.6 -> 81.8 | Improved |
| use bi-modal attention head instead of FC head (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 79.5 -> 81.8 | Improved |
| use bi-modal attention head instead of KNN head (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 79.9 -> 81.8 | Improved |
| use text selection ruler (TSR) instead of random sentence selection (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 80.0 -> 81.8 | Improved |
| use TSR salient sentences instead of basic prompts only (ablation) | VLG ViT-B/16 · Kinetics-Close ablation | top-1 accuracy: 78.9 -> 81.8 | Improved |
| increase temporal module layers from 0 to 6 (ablation) | VLG ViT-B/16 · Kinetics-Close | top-1 accuracy: 79.8 -> 81.8 | Improved |
| increase temporal module layers from 6 to 8 (ablation) | VLG ViT-B/16 · Kinetics-Close | top-1 accuracy: 81.8 -> 80.6 | Hurt |
| add temporal module layers (0 to 8) in few-shot setting (ablation) | VLG ViT-B/16 · Kinetics-Fewshot | top-1 accuracy: 96.9 -> 95.2 | Hurt |
| add logits distillation with 1-alpha=0.5 (ablation) | VLG ResNet-50 · Kinetics-LT | overall top-1 accuracy: 57.5 -> 60.8 | Improved |
| raise logits distillation weight 1-alpha to 0.9 (ablation) | VLG ResNet-50 · Kinetics-LT | overall top-1 accuracy: 60.8 -> 54.7 | Hurt |
| use feature distillation (1-alpha=0.5) instead of logits distillation (ablation) | VLG ResNet-50 · Kinetics-LT | overall top-1 accuracy: 60.8 -> 59.7 | Hurt |
Closest papers by meaning
- Prompting Visual-Language Models for Efficient Video Understanding (ECCV 2022 · similarity 0.91)
- OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition (CVPR 2024 · similarity 0.91)
- MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge (ICCV 2023 · similarity 0.90)
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.88)
- MoTE: Reconciling Generalization with Specialization for Visual-Language to Video Knowledge Transfer (NeurIPS 2024 · similarity 0.88)
- VicTR: Video-conditioned Text Representations for Activity Recognition (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
