AI Paper Atlas

Research map / Video / Video-language understanding

SViTT: Temporal Learning of Sparse Video-Text Transformers

Yi Li, Kyle Min, Subarna Tripathi, Nuno Vasconcelos

CVPR 2023 · Video › Video-language understanding · approach: Video-language pretraining · 19 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use hybrid edge and node sparsity instead of dense attentionSViTT · DiDeMo zero-shot retrievalRecall mean: 49.3 -> 51.5Improved
Use hybrid edge and node sparsity instead of dense attentionSViTT · MSR-VTT zero-shot retrievalRecall mean: 43.6 -> 43.8No clear effect
Use hybrid edge and node sparsity instead of dense attentionSViTT · Charades retrieval, fine-tunedRecall mean: 32.7 -> 35.7Improved
Use hybrid edge and node sparsity instead of dense attentionSViTT · SSv2-Label retrieval, fine-tunedRecall mean: 66.1 -> 69.3Improved
Use hybrid edge and node sparsity instead of dense attentionSViTT · ActivityNet-QAaccuracy: 42.3 -> 43.2Improved
Add text-guided token pruning at q_m=0.1 after visual pruning (ablation)Dense video-text model · DiDeMo zero-shot retrievalRecall mean: 46.7 -> 47.9Improved
Use edge sparsity with (1, 5, 56) instead of (1, 3, 56) (ablation)Dense video-text model · DiDeMo retrievalRecall mean: 37.6 -> 43.8Improved
Remove the single local diagonal-attention block (ablation)Sparse video-text model · DiDeMo retrievalRecallHurt
Increase the local attention window to neighboring blocks (ablation)Sparse video-text model · DiDeMo retrievalRecallHurt
Train using temporal sparse expansion rather than single-stage training (ablation)SViTT · DiDeMo, 16 framesRecall mean: 47.7 -> 52.2Improved
Reorder tokens with Morton or Hilbert curves instead of standard flattening (ablation)SViTT · MSR-VTT retrievalRecall mean: 38.5 -> 36.9Hurt
Evaluate with shuffled rather than normal frame order (ablation)SViTT Hybrid · SSv2-Label retrievalRecall: 69.3 -> 65.8Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy