Research map / Video / Video-language understanding
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Max Bain, Arsha Nagrani, Gül Varol, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add large-scale pretraining on CC3M+WebVid2M versus no pretraining (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 5.6 -> 27.3 | Improved |
| pretrain on WebVid2M instead of HowTo-17M subset (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 24.1 -> 26.0 | Improved |
| add image data (CC3M) to WebVid2M video pretraining (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 26.0 -> 27.3 | Improved |
| temporal expansion at pretraining rather than at finetuning (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 24.9 -> 26.0 | Improved |
| add 1 to 4 frame curriculum at pretraining (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 26.0 -> 26.6 | Improved |
| add 1 to 4 to 8 frame curriculum instead of 8 frames from start (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 25.4 -> 27.4 | Improved |
| use 4 frames instead of a single frame (ablation) | space-time transformer · MSR-VTT 1K-A | R@1: 18.8 -> 24.9 | Improved |
| replace ResNet-101 with space-time transformer backbone (ablation) | video encoder · MSR-VTT test | R@1: 11.5 -> 26.8 | Improved |
| use distilbert instead of t5-base text backbone (ablation) | text encoder · MSR-VTT test | R@1: 24.0 -> 26.8 | Improved |
| use bert-base instead of distilbert text backbone (ablation) | text encoder · MSR-VTT test | R@1: 26.8 -> 27.5 | No clear effect |
| use the modified space-time attention block instead of divided block (ablation) | space-time attention · MSR-VTT 1K-A (zero-shot) | R@1: 13.0 -> 14.6 | Improved |
| choice of temporal embedding expansion (zero-pad vs bilinear) (ablation) | temporal expansion 4 to 8 frames · MSR-VTT 1K-A | R@1: 30.7 vs 28.3 | No clear effect |
Closest papers by meaning
- Thinking Fast and Slow: Efficient Text-to-Visual Retrieval With Transformers (CVPR 2021 · similarity 0.92)
- TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval (ICLR 2025 · similarity 0.91)
- Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning (AAAI 2024 · similarity 0.90)
- Mitigating Semantic Collapse in Partially Relevant Video Retrieval (NeurIPS 2025 · similarity 0.89)
- Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval (CVPR 2025 · similarity 0.89)
- Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
