Research map / Vision-language and multimodal / CLIP and zero-shot
Revisiting Classifier: Transferring Vision-Language Models for Video Recognition
Wenhao Wu, Zhun Sun, Wanli Ouyang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace a randomly initialized learnable head with a frozen CLIP text-embedding classifier | CLIP ViT-B/16 video recognition · Kinetics-400, zero-shot | accuracy: 0.2 -> 54.2 | Improved |
| Replace a randomly initialized learnable head with a frozen CLIP text-embedding classifier | CLIP ViT-B/16 video recognition · Kinetics-400, 2-shot | accuracy: 21.6 -> 65.3 | Improved |
| Replace a randomly initialized learnable head with a frozen CLIP text-embedding classifier | CLIP ViT-B/16 video recognition · Kinetics-400, full-shot | accuracy: 75.3 -> 80.1 | Improved |
| Add semantic text embeddings as the frozen classifier rather than random normal weights (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 59.3 -> 81.5 | Improved |
| Add semantic text embeddings as the frozen classifier rather than orthogonal weights (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 59.4 -> 81.5 | Improved |
| Use CLIP text embeddings instead of LDA visual-statistic classifier weights (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 80.8 -> 81.5 | Improved |
| Use DistilBERT rather than CLIP to produce the text classifier (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 81.5 -> 81.4 | No clear effect |
| Replace TAP with temporal transformer modeling (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 80.1 -> 81.5 | Improved |
| Add temporal 1D convolution to the global class token (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 80.1 -> 80.4 | Improved |
| Use temporal 1D convolutions in ResNet stages (ablation) | CLIP ResNet-50 video recognition · Kinetics-400 | Top-1 accuracy: 71.2 -> 67.2 | Hurt |
| Use multiple fixed templates rather than one fixed template (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 81.5 -> 80.9 | Hurt |
| Use a learnable prompt template rather than one fixed template (ablation) | CLIP ViT-B/16 video recognition · Kinetics-400 | Top-1 accuracy: 81.5 -> 81.2 | No clear effect |
Closest papers by meaning
- Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective (IJCV 2023 · similarity 0.96)
- AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation (NeurIPS 2024 · similarity 0.90)
- Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages (CVPR 2025 · similarity 0.89)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.88)
- Making Large Vision Language Models to Be Good Few-Shot Learners (AAAI 2025 · similarity 0.88)
- Why are Visually-Grounded Language Models Bad at Image Classification? (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
