Research map / Vision-language and multimodal / CLIP and zero-shot
SAMPLE: Semantic Alignment through Temporal-Adaptive Multimodal Prompt Learning for Event-Based Open-Vocabulary Action Recognition
Jing Wang, Rui Zhao, Ruiqin Xiong, Xingtao Wang, Xiaopeng Fan, Tiejun Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Event-Text multimodal prompting to frozen CLIP (ablation) | CLIP · SeAct | Top-1 accuracy: 9.85 -> 65.25 | Improved |
| add Temporal-Adaptive prompting to frozen CLIP (ablation) | CLIP · SeAct | Top-1 accuracy: 9.85 -> 67.41 | Improved |
| add Temporal-Adaptive prompting on top of Event-Text prompting (ablation) | SAMPLE · SeAct | Top-1 accuracy: 65.25 -> 69.79 | Improved |
| add Event-Text prompting on top of Temporal-Adaptive prompting (ablation) | SAMPLE · SeAct | Top-1 accuracy: 67.41 -> 69.79 | Improved |
| add Temporal-Adaptive prompting on top of Event-Text prompting (ablation) | SAMPLE · SeAct | Top-5 accuracy: 92.34 -> 97.83 | Improved |
| add Temporal-Adaptive prompting (extra parameters) on top of Event-Text prompting (ablation) | SAMPLE · SeAct | trainable parameters (M): 4.290 -> 8.681 | Mixed |
| vary prompt insertion depth J (ablation) | SAMPLE · SeAct | Top-1 accuracy: zero-shot peaks at J = 8, fully-supervised at J = 12 | Mixed |
| prompt learning with frozen CLIP instead of fine-tuning the event encoder (SAMPLE vs ExACT) | SAMPLE · zero-shot SeAct (trained on HARDVS) | Top-1 accuracy: 11.54 -> 35.71 | Improved |
| prompt learning instead of ExACT fine-tuning | SAMPLE · DVS128Gesture zero-shot | Top-1 accuracy: 15.91 -> 55.17 | Improved |
| prompt learning instead of ExACT fine-tuning | SAMPLE · base-to-novel HARDVS novel accuracy | novel accuracy: 35.14 -> 63.22 | Improved |
| prompt learning instead of best prior method at 2 shots | SAMPLE · DVS128Gesture few-shot K=2 | Top-1 accuracy: 13.85 vs 18.92 (M2-CLIP) | Hurt |
| prompt learning instead of best prior method at 2 shots | SAMPLE · SeAct few-shot K=2 | Top-1 accuracy: 20.34 -> 33.33 | Improved |
Later papers that compared with this method
1 later accepted paper reports Sample as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Rethinking Token Reduction for Large Vision-Language Models (CVPR 2026 · better on 7 of 7 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Does Video-Text Pretraining Help Open-Vocabulary Online Action Detection? (NeurIPS 2024 · similarity 0.88)
- OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies (CVPR 2024 · similarity 0.87)
- Align Before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognition (CVPR 2024 · similarity 0.87)
- Weakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- Kronecker Mask and Interpretive Prompts are Language-Action Video Learners (ICLR 2025 · similarity 0.87)
- Temporal Alignment-Free Video Matching for Few-shot Action Recognition (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
