Research map / Video / Video-language understanding
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-Text Retrieval
Yuying Ge, Yixiao Ge, Xihui Liu, Jinpeng Wang, Jianping Wu, Ying Shan, Xiaohu Qie, Ping Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add masked visual modeling with text-aligned feature targets (ablation) | MILES pre-training · MSR-VTT text-to-video (zero-shot) | R@1: 21.9 -> 26.1 | Improved |
| use aligned features instead of pixel targets (ablation) | MILES MVM · MSR-VTT text-to-video (zero-shot) | R@1: 22.9 -> 26.1 | Improved |
| use aligned features instead of discrete token targets (ablation) | MILES MVM · MSR-VTT text-to-video (zero-shot) | R@1: 24.1 -> 26.1 | Improved |
| add a joint encoder when targets are aligned features (ablation) | MILES MVM · MSR-VTT text-to-video (zero-shot) | R@1: 26.1 -> 25.2 | Hurt |
| add a joint encoder with pixel targets (ablation) | MILES MVM · MSR-VTT text-to-video (zero-shot) | R@1: 22.9 -> 24.4 | Improved |
| update snapshot encoder from previous epoch with momentum rather than current iteration (ablation) | MILES snapshot encoder · MSR-VTT text-to-video (zero-shot) | R@1: 25.0 -> 26.1 | Improved |
| add momentum update to epoch-level snapshot (ablation) | MILES snapshot encoder · MSR-VTT text-to-video (zero-shot) | R@1: 23.8 -> 26.1 | Improved |
| use tube masking (ablation) | MILES masking · MSR-VTT text-to-video (zero-shot) | R@1: 24.9 -> 26.1 | Improved |
| use block-wise instead of random spatial masking (ablation) | MILES masking · MSR-VTT text-to-video (zero-shot) | R@1: 25.0 -> 26.1 | Improved |
| use block masking ratio 75% instead of 85% (ablation) | MILES masking · MSR-VTT text-to-video (zero-shot) | R@1: 25.8 -> 26.1 | Improved |
| use spatial patch masking instead of frame-wise masking (ablation) | MILES masking · MSR-VTT text-to-video (zero-shot) | R@1: 25.3 -> 26.1 | Improved |
| add MVM during MSR-VTT fine-tuning (ablation) | MILES fine-tuning · MSR-VTT text-to-video (fine-tuned) | R@1: 37.7 -> 37.8 | No clear effect |
Later papers that compared with this method
3 later accepted papers report Miles as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- EA-VTR: Event-Aware Video-Text Retrieval (ECCV 2024 · better on 5 of 5 shared results)
- Learning Trajectory-Word Alignments for Video-Language Tasks (ICCV 2023 · better on 3 of 3 shared results)
- Exploring Temporal Concurrency for Video-Language Representation Learning (ICCV 2023 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval (ICCV 2021 · similarity 0.89)
- Cluster-Wise Spatio-Temporal Masking for Efficient Video-Language Pretraining (CVPR 2026 · similarity 0.89)
- Video Language Model Pretraining with Spatio-temporal Masking (CVPR 2025 · similarity 0.87)
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.87)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.86)
- A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
