Research map / Video / Video-language understanding
ECLIPSE: Efficient Long-Range Video Retrieval Using Sight and Sound
Yan-Bo Lin, Jie Lei, Mohit Bansal, Gedas Bertasius
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio via audiovisual attention to CLIP4Clip | CLIP4Clip · ActivityNet Captions (val1) | T2V R@1: 40.7 -> 42.3 | Improved |
| replace 96 video frames with 32 frames plus audio | CLIP4Clip · ActivityNet Captions (val1) | efficiency: 1251 -> 827 GFLOPs, 17.39 -> 50.93 samples/sec, 41.7 -> 42.3 R@1 | Improved |
| add audio on top of video (vs 64-frame video-only) (ablation) | ECLIPSE · ActivityNet Captions (val1) | T2V R@1: 40.7 -> 42.3 (audio-only 2.7) | Improved |
| use ViT-B/16 instead of ViT-B/32 (ablation) | ECLIPSE · ActivityNet Captions (val1) | T2V R@1: 42.3 -> 45.3 | Improved |
| use dual-pathway A2V+V2A attention instead of A2V only or joint AV (ablation) | ECLIPSE · ActivityNet Captions (val1) (8 frames) | T2V R@1 | Improved |
| use 30 s audio instead of 10 s (ablation) | ECLIPSE · ActivityNet Captions (val1) | T2V R@1: +0.5% R@1 | Improved |
| change the audio encoder (VGGish, VGGSound, AST) (ablation) | ECLIPSE · ActivityNet Captions (val1) | T2V R@1 | No clear effect |
| insert audiovisual block in all 12 layers instead of fewer (ablation) | ECLIPSE · ActivityNet Captions (val1) | T2V R@1 | Improved |
| CLIP pretraining instead of ImageNet-21k pretraining (ablation) | ECLIPSE · ActivityNet, DiDeMo, YouCook2, QVHighlights | T2V R@1: 27.1%, 34.6%, 24.2%, 42.5% better | Improved |
| random instead of uniform frame sampling at 32 frames (ablation) | CLIP4Clip (video-only) · ActivityNet Captions (val1) | T2V R@1: 40.4 -> 41.0 | Improved |
| random instead of uniform frame sampling at 96 frames (ablation) | CLIP4Clip (video-only) · ActivityNet Captions (val1) | T2V R@1: 41.7 -> 40.9 | Hurt |
| remove overlapping VGGSound test videos from ActivityNet (ablation) | ECLIPSE and CLIP4Clip · ActivityNet Captions (val1) | T2V R@1: 42.3 -> 42.3 and 40.7 -> 40.8 | No clear effect |
Closest papers by meaning
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.86)
- From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos (NeurIPS 2025 · similarity 0.86)
- DrVideo: Document Retrieval Based Long Video Understanding (CVPR 2025 · similarity 0.85)
- Learning Audio-guided Video Representation with Gated Attention for Video-Text Retrieval (CVPR 2025 · similarity 0.85)
- MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (NeurIPS 2025 · similarity 0.85)
- AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
