Research map / Video / Video-language understanding
Learning Transferable Spatiotemporal Representations From Natural Script Knowledge
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add transcript sorting (Lsort) to the video-transcript contrastive objective (ablation) | TVTS pre-training · SSV2 | top-1 accuracy: 67.0 (↓1.5) -> 68.5 | Helped |
| Add transcript sorting (Lsort) to the video-transcript contrastive objective (ablation) | TVTS pre-training · Kinetics-400 | top-1 accuracy: 77.8 (↓1.0) -> 78.8 | Helped |
| Allow Lsort gradients to update transcript representations (ablation) | TVTS pre-training · SSV2 | top-1 accuracy: 66.2 (↓2.3) -> 68.5 | Helped |
| Allow Lsort gradients to update transcript representations (ablation) | TVTS pre-training · Kinetics-400 | top-1 accuracy: 76.5 (↓2.3) -> 78.8 | Helped |
| Use K-way classification instead of pairwise transcript-order prediction (ablation) | TVTS pre-training · SSV2 | top-1 accuracy: 67.4 (↓1.1) -> 68.5 | Helped |
| Use K-way classification instead of pairwise transcript-order prediction (ablation) | TVTS pre-training · Kinetics-400 | top-1 accuracy: 78.1 (↓0.7) -> 78.8 | Helped |
| Use K-way classification instead of K!-way transcript-order classification (ablation) | TVTS pre-training · SSV2 | top-1 accuracy: 67.2 (↓1.3) -> 68.5 | Helped |
| Use K-way classification instead of K!-way transcript-order classification (ablation) | TVTS pre-training · Kinetics-400 | top-1 accuracy: 78.0 (↓0.8) -> 78.8 | Helped |
| Use transcript sorting rather than only the video-transcript contrastive objective (ablation) | TVTS pre-training · UCF-101 and HMDB-51 linear probing | top-1 accuracy: 81.2 -> 83.4; 56.5 -> 58.4 | Helped |
| Use transcript sorting rather than frame or video ordering as the pretext task (ablation) | YT-Temporal pre-training · UCF-101 and HMDB-51 linear probing | top-1 accuracy: 74.9 -> 83.4; 49.6 -> 58.4 | Helped |
| Post-pre-train with CC3M and WebVid-2M using the contrastive objective | TVTS · SSV2 fine-tuning | top-1 accuracy: 68.5 -> 69.1 | Helped |
| Post-pre-train with CC3M and WebVid-2M using the contrastive objective | TVTS · Kinetics-400 fine-tuning | top-1 accuracy: 78.8 -> 79.8 | Helped |
Closest papers by meaning
- DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences (NeurIPS 2026 · similarity 0.89)
- V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models (NeurIPS 2026 · similarity 0.88)
- SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding (CVPR 2025 · similarity 0.87)
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.87)
- VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM (CVPR 2025 · similarity 0.87)
- Look, Remember and Reason: Grounded Reasoning in Videos with Language Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.