Research map / Video / Action recognition
Hierarchical Self-Supervised Representation Learning for Movie Understanding
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pretrain TxE on LVU with event-level mask prediction instead of leaving TxE unpretrained | Slow-D+TxE+TxD with CVRL-K400 · VidSitu semantic role prediction | CIDEr: 54.40 -> 61.18 | Helped |
| Pretrain the backbone with CVRL instead of supervised K400 pretraining | Slow-D+TxE+TxD without TxE pretraining · VidSitu semantic role prediction | CIDEr: 51.36 -> 54.40 | Helped |
| Pretrain the backbone on K400 and TxE on VidSitu, rather than both levels on VidSitu | Slow-D+TxE+TxD · VidSitu semantic role prediction | CIDEr: 44.32 -> 60.34 | Helped |
| Pretrain TxE with event-level mask prediction rather than leave it randomly initialized | Slow-D+TxE · VidSitu event relation prediction | Mean-Acc: 25.00 ± 0.00 -> 35.32 ± 0.17 | Helped |
| Pretrain the backbone with MoDist rather than supervised K400 labels | Slow-D · VidSitu verb prediction | Acc@1: 38.29 -> 42.96 | Helped |
| Add VidSitu self-supervised pretraining after K400 pretraining | Slow-D with MoDist · VidSitu verb prediction | Acc@1: 42.96 -> 44.67 | Helped |
| Add contextualized scene features to OT instance features | OT · LVU (9 tasks) | mean rank: 3.22 -> 2.33 | Helped |
| Use scene features from the pretrained contextualizer rather than supervised, non-contextualized Slow-D features | LVU scene pathway · LVU (9 tasks) | mean rank: 4.44 -> 2.89 | Helped |
| Use mask sizes {1, 2} rather than {1, 2, 3} (ablation) | TxE mask prediction · VidSitu semantic role prediction | CIDEr: 61.18 ± 1.48 -> 59.15 ± 2.01 | Hurt |
| Use mask sizes {1, 2, 3, 4} rather than {1, 2, 3} (ablation) | TxE mask prediction · VidSitu semantic role prediction | CIDEr: 61.18 ± 1.48 -> 59.45 ± 1.31 | Hurt |
| Use 1s token stride rather than 2s (ablation) | TxE mask prediction · VidSitu semantic role prediction | CIDEr: 61.18 ± 1.48 -> 60.15 ± 1.13 | Hurt |
| Replace contrastive mask-prediction loss with L2 distance loss (ablation) | TxE mask prediction · VidSitu semantic role prediction | CIDEr: 61.18 ± 1.48 -> 58.88 ± 1.13 | Hurt |
Closest papers by meaning
- Self-Supervised Video Representation Learning by Context and Motion Decoupling (CVPR 2021 · similarity 0.90)
- No More Shortcuts: Realizing the Potential of Temporal Self-Supervision (AAAI 2024 · similarity 0.89)
- Self-supervised Co-Training for Video Representation Learning (NeurIPS 2020 · similarity 0.88)
- Frame-Wise Action Representations for Long Videos via Sequence Contrastive Learning (CVPR 2022 · similarity 0.87)
- Self-Supervised Learning of Compressed Video Representations (ICLR 2021 · similarity 0.87)
- Self-Supervised Learning of Video-Induced Visual Invariances (CVPR 2020 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.