Research map / Video / Video-language understanding
Self-supervised video pretraining yields robust and more human-aligned visual representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove temporal deformations (T=0) (ablation) | VITO · PASCAL segmentation | mIoU: 74.8 -> 76.3 | Helped |
| remove temporal deformations (T=0) (ablation) | VITO · Human error consistency | error consistency: 0.323 -> 0.422 | Helped |
| use attention pooling on single-scale model (ablation) | VITO · PASCAL segmentation | mIoU: 75.2 -> 75.4 | Helped |
| use two scales instead of one (no attention) (ablation) | VITO · PASCAL segmentation | mIoU: 75.2 -> 75.8 | Helped |
| combine two scales with attention pooling (ablation) | VITO · PASCAL segmentation | mIoU: 75.4 -> 76.3 | Helped |
| pretrain on AudioSet instead of curated VideoNet (ablation) | VITO · PASCAL segmentation | mIoU: 76.3 -> 73.8 | Hurt |
| replace VITO with MoCLR on VideoNet (ablation) | VideoNet pretraining · UCF101 | top-1: 87.4 -> 83.0 | Hurt |
| replace VITO with MoCLR on VideoNet (ablation) | VideoNet pretraining · ImageNet-A | top-1: 5.4 -> 2.3 | Hurt |
| increase minimum crop scale from 0.08 to 0.4 (ablation) | VITO · PASCAL segmentation | mIoU: +1.7% mIoU | Helped |
| lengthen temporal clip from T=1.28s to T=2.56s (ablation) | VITO · PASCAL segmentation | mIoU: +1.0% mIoU | Helped |
| lengthen temporal clip from T=2.56s to T=3.2s (ablation) | VITO · PASCAL segmentation | mIoU | No clear effect |
| pretrain MoCLR on frames of Kinetics700 instead of ImageNet (ablation) | MoCLR · PASCAL segmentation | mIoU: -6.8% mIoU | Hurt |
Closest papers by meaning
- Evaluating alignment between humans and neural network representations in image-based learning tasks (NeurIPS 2024 · similarity 0.88)
- Perception Encoder: The best visual embeddings are not at the output of the network (NeurIPS 2025 · similarity 0.88)
- Video In-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators (ICLR 2025 · similarity 0.87)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.87)
- VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.87)
- No More Shortcuts: Realizing the Potential of Temporal Self-Supervision (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.