Research map / Video / Video-language understanding
Prompting Visual-Language Models for Efficient Video Understanding
Chen Ju, Tengda Han, Kunhao Zheng, Ya Zhang, Weidi Xie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase learnable prompt vectors from 2+X+2 to 16+X+16 (ablation) | CLIP prompt learning · Kinetics-400 closed-set | TOP1 accuracy: 65.4 -> 68.8 | Improved |
| add one temporal Transformer layer on frame features (ablation) | CLIP prompt learning (16+X+16) · Kinetics-400 closed-set | TOP1 accuracy: 68.8 -> 75.8 | Improved |
| go from 2 to 4 temporal Transformer layers (ablation) | CLIP prompt learning (16+X+16) · Kinetics-700 closed-set | average of TOP1 and TOP5: 76.6 -> 76.4 | No clear effect |
| learn prompt vectors instead of handcrafted template (ablation) | CLIP · Kinetics-700 closed-set | average of TOP1 and TOP5: 52.4 -> 70.5 | Improved |
| replace linear probe with learned prompts (16K parameters) (ablation) | CLIP image encoder · Kinetics-700 closed-set | average of TOP1 and TOP5: 66.1 -> 70.5 | Improved |
| add 4-layer temporal Transformer to prompted CLIP (ablation) | CLIP prompt learning · Something-Something V2 closed-set | TOP1 accuracy: 18.4 -> 38.1 | Improved |
| add temporal Transformer in 5-shot 5-way few-shot recognition (ablation) | CLIP prompt learning · UCF-101 few-shot 5-way | TOP1 accuracy: 98.3 -> 97.8 | No clear effect |
| add temporal Transformer in 5-shot all-class few-shot recognition (ablation) | CLIP prompt learning · UCF-101 few-shot 5-shot, all classes | TOP1 accuracy: 77.6 -> 79.5 | Improved |
| learn prompts instead of handcrafted template in few-shot (ablation) | CLIP · UCF-101 few-shot 5-shot, all classes | TOP1 accuracy: 64.7 -> 77.6 | Improved |
| add temporal Transformer layers for zero-shot recognition (ablation) | CLIP prompt learning (16+X+16) · Kinetics-700 zero-shot | TOP1 accuracy: 58.4 -> 45.5 | Hurt |
| increase prompt vectors from 4+X+4 to 32+X+32 in zero-shot recognition (ablation) | CLIP prompt learning · Kinetics-700 zero-shot | TOP1 accuracy: 57.4 -> 57.5 | No clear effect |
| learn prompts instead of handcrafted template for zero-shot recognition (ablation) | CLIP · Kinetics-700 zero-shot | TOP1 accuracy: 52.4 -> 58.4 | Improved |
Closest papers by meaning
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.91)
- Align Before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognition (CVPR 2024 · similarity 0.90)
- Kronecker Mask and Interpretive Prompts are Language-Action Video Learners (ICLR 2025 · similarity 0.90)
- OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition (CVPR 2024 · similarity 0.90)
- Weakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
