Research map / Vision-language and multimodal / CLIP and zero-shot
MAtch, eXpand and Improve: Unsupervised Finetuning for Zero-Shot Action Recognition with Language Knowledge
Wei Lin, Leonid Karlinsky, Nina Shvetsova, Horst Possegger, Mateusz Kozinski, Rameswar Panda, Rogerio Feris, Hilde Kuehne, Horst Bischof
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune CLIP on unlabeled K400 with MAXI (K400 dict only) | CLIP ViT-B/16 · HMDB51 zero-shot | top-1 accuracy: 38.0 -> 50.5 | Improved |
| add GPT-3 verbs to the text bag (ablation) | MAXI · UCF101 zero-shot | top-1 accuracy: 76.6 -> 77.8 | Improved |
| add BLIP verbs on top of GPT-3 verbs (ablation) | MAXI · UCF101 zero-shot | top-1 accuracy: 76.45 -> 77.88 | Improved |
| use BLIP verbs instead of BLIP object nouns (ablation) | MAXI text bag · HMDB51 zero-shot | top-1 accuracy: 50.15 -> 50.92 | Improved |
| filter text bags to top 90% by CLIP similarity (ablation) | MAXI · HMDB51 zero-shot | top-1 accuracy: 51.09 -> 52.24 | Improved |
| aggressive filtering to 30% of videos (ablation) | MAXI · HMDB51 zero-shot | top-1 accuracy: 51.09 -> 47.73 | Hurt |
| use MIL-NCE instead of cross entropy over fixed classes (ablation) | MAXI · K600 zero-shot | top-1 accuracy: 65.09 -> 71.24 | Improved |
| add same-best-matched-text positives to MIL-NCE (ablation) | MAXI · K600 zero-shot | top-1 accuracy: 70.14 -> 71.24 | Improved |
| increase text bag size from 1 to 16 (ablation) | MAXI · HMDB51 zero-shot | top-1 accuracy: 49.85 -> 51.09 | Improved |
| use MAXI instead of CLIP initialization for few-shot (2-shot UCF101) | ViFi-CLIP few-shot · UCF101 2-shot | accuracy: 84.0 -> 86.8 | Improved |
| use under-specified (200-class) action dictionary (ablation) | MAXI · UCF101 zero-shot | top-1 accuracy: 78.18 -> 75.10 | Hurt |
| use over-specified dictionary with 400 random added verbs (ablation) | MAXI · HMDB51 zero-shot | top-1 accuracy: 50.35 -> 45.97 | Hurt |
Closest papers by meaning
- VideoNet: A Large-Scale Dataset for Domain-Specific Action Recognition (CVPR 2026 · similarity 0.91)
- Video Action Recognition with Attentive Semantic Units (ICCV 2023 · similarity 0.91)
- Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning (CVPR 2025 · similarity 0.90)
- OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition (CVPR 2024 · similarity 0.89)
- PeVL: Pose-Enhanced Vision-Language Model for Fine-Grained Human Action Recognition (CVPR 2024 · similarity 0.88)
- Part-aware Unified Representation of Language and Skeleton for Zero-shot Action Recognition (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
