Research map / Video / Video-language understanding
Connecting Vision and Language With Video Localized Narratives
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add COCO-PNG pre-training to UVO-VNG training (ablation) | ReferFormer-VNG on OVIS-VNG · OVIS-VNG | J&F: 26.2 -> 32.0 | Helped |
| Add UVO-VNG training after COCO-PNG pre-training (ablation) | ReferFormer-VNG on UVO-VNG · UVO-VNG | J&F: 39.6 -> 46.4 | Helped |
| Fine-tune on OVIS-VNG after COCO-PNG and UVO-VNG training (ablation) | ReferFormer-VNG · OVIS-VNG | J&F: 32.0 -> 32.7 | Helped |
| Replace full narrative input with only the target noun (ablation) | ReferFormer on OVIS-VNG · OVIS-VNG | J&F: 22.9 -> 25.7 | Helped |
| Replace full narrative input with only the target noun (ablation) | ReferFormer on UVO-VNG · UVO-VNG | J&F: 25.8 -> 35.6 | Helped |
| Use three equally spaced video frames instead of one middle frame (ablation) | PaLI-1.5B zero-shot · Oops-QA text-output test | exact match accuracy: 24.1% -> 25.1% | Helped |
| Use three equally spaced video frames instead of one middle frame (ablation) | Fine-tuned PaLI-1.5B · Oops-QA text-output test | exact match accuracy: 44.9% -> 49.0% | Helped |
| Fine-tune on the Oops-QA training set (ablation) | PaLI-1.5B with one middle frame · Oops-QA text-output test | exact match accuracy: 24.1% -> 44.9% | Helped |
| Use Swin-Large initialized on ImageNet-21k instead of ResNet-50 initialized on ImageNet-1k (ablation) | ReferFormer-VNG with COCO-PNG pre-training and no UVO-VNG training · UVO-VNG | J&F: 39.6 -> 55.1 | Helped |
| Train the visual backbone directly on OVIS-VNG without other VNG-related training data (ablation) | ReferFormer-VNG, ResNet-50 · OVIS-VNG | J&F: 25.1 vs 32.4 | Hurt |
| Filter out disconnected mouse trace segments (ablation) | Video Localized Narratives trace localization · OVIS-VNG | trace precision on segmentation masks: 73.2% -> 77.3% | Helped |
Closest papers by meaning
- HourVideo: 1-Hour Video-Language Understanding (NeurIPS 2024 · similarity 0.88)
- VideoGLaMM : A Large Multimodal Model for Pixel-Level Visual Grounding in Videos (CVPR 2025 · similarity 0.88)
- ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models (ICLR 2024 · similarity 0.88)
- SAMA: Towards Multi-Turn Referential Grounded Video Chat with Large Language Models (NeurIPS 2025 · similarity 0.88)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.88)
- Unleashing Hour-Scale Video Training for Long Video-Language Understanding (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.