Research map / Video / Action recognition
Long-Short Temporal Contrastive Learning of Video Transformers
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain ST Swin with LSTCL rather than training from scratch | ST Swin · Kinetics-400 | Top-1 accuracy: 71.1 -> 79.8 | Helped |
| Pretrain ST Swin with LSTCL rather than ImageNet-1K supervision | ST Swin · Kinetics-400 | accuracy: 76.0 -> 79.8 | Helped |
| Pretrain ST Swin with LSTCL rather than training from scratch | ST Swin · Kinetics-600 | Top-1 accuracy: 74.7 -> 82.0 | Helped |
| Pretrain ST Swin with LSTCL rather than training from scratch | ST Swin · Something-Something-V2 | Top-1 accuracy: 38.4 -> 64.8 | Helped |
| Pretrain ST Swin with LSTCL on Kinetics-400 | ST Swin · UCF101 | Accuracy: +0.5 | Helped |
| Pretrain ST Swin with LSTCL on Kinetics-400 | ST Swin · HMDB51 | Accuracy: +0.9 | Helped |
| Use long and short views with different temporal extents (τL=32 rather than τL=τS=8) (ablation) | LSTCL with BYOL · Kinetics-400 | Accuracy: +2.6% | Helped |
| Use long and short views with different temporal extents (τL=32 rather than τL=τS=8) (ablation) | LSTCL with MoCo v3 · Kinetics-400 | Accuracy: +3.1% | Helped |
| Use long and short views with different temporal extents (τL=32 rather than τL=τS=8) (ablation) | LSTCL with SimSiam · Kinetics-400 | Accuracy: +1.6% | Helped |
| Randomly sample both temporal strides instead of fixing τS=8 and τL=32 (ablation) | LSTCL with MoCo v3 · Kinetics-400 | Accuracy: 76.6 -> 75.6 | Hurt |
| Sample the short clip within the long clip rather than independently (ablation) | LSTCL with MoCo v3 · Kinetics-400 | Accuracy: 76.6 -> 76.2 | Hurt |
| Require long and short clips to be disjoint rather than sampling independently (ablation) | LSTCL with MoCo v3 · Kinetics-400 | Accuracy: 76.6 -> 72.6 | Hurt |
Closest papers by meaning
- Cross-Architecture Self-Supervised Video Representation Learning (CVPR 2022 · similarity 0.91)
- Time Is MattEr: Temporal Self-supervision for Video Transformers (ICML 2022 · similarity 0.90)
- Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · similarity 0.89)
- No More Shortcuts: Realizing the Potential of Temporal Self-Supervision (AAAI 2024 · similarity 0.88)
- Understanding Video Transformers via Universal Concept Discovery (CVPR 2024 · similarity 0.87)
- A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.