Research map / Video / Video-language understanding
Video OWL-ViT: Temporally-consistent Open-world Localization in Video
Georg Heigold, Matthias Minderer, Alexey Gritsenko, Alex Bewley, Daniel Keysers, Mario Lučić, Fisher Yu, Thomas Kipf
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add end-to-end video decoder instead of appearance-matching tracking-by-detection (ablation) | OWL-ViT · TAO-OW unknown | OWTA: 31.0 -> 45.4 | Improved |
| add transformer decoder to OWL-ViT (encoder-only to enc-dec) (ablation) | OWL-ViT L/14 · LVIS rare | AP rare: 31.8 -> 28.9 | Hurt |
| train on 4-frame clips instead of 2-frame clips (ablation) | Video OWL-ViT · TAO-OW unknown | OWTA: 35.2 -> 45.3 | Improved |
| extend clip length from 4 to 6 frames (ablation) | Video OWL-ViT · TAO-OW unknown | OWTA: 45.3 -> 45.4 | No clear effect |
| train on pseudo-videos from images instead of only real videos (ablation) | Video OWL-ViT · TAO-OW unknown | OWTA: 33.9 -> 45.9 | Improved |
| combine real and pseudo-videos instead of pseudo-videos only (ablation) | Video OWL-ViT · TAO-OW known vs unknown | OWTA: known 56.9 -> 59.0; unknown 45.9 -> 45.4 | Mixed |
| add score calibration (per-track objectness threshold) (ablation) | Video OWL-ViT · TAO-OW unknown short tracks | OWTA: 12.5 -> 22.2 | Improved |
| add temporal video mosaic augmentation (ablation) | Video OWL-ViT · TAO-OW known short tracks | OWTA: 26.1 -> 31.3 | Improved |
| increase inference frame rate from 1 to 4 FPS (ablation) | Video OWL-ViT · TAO-OW | OWTA: known 59.0 -> 61.1; unknown 45.4 -> 46.1 | Improved |
| pre-train whole Enc-dec model on detection before video training (ablation) | Video OWL-ViT · TAO-OW | OWTA | No clear effect |
Closest papers by meaning
- Learning Object State Changes in Videos: An Open-World Perspective (CVPR 2024 · similarity 0.89)
- Recognize Any Regions (NeurIPS 2024 · similarity 0.87)
- RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations (CVPR 2025 · similarity 0.87)
- Re-thinking Temporal Search for Long-Form Video Understanding (CVPR 2025 · similarity 0.87)
- OW-VISCapTor: Abstractors for Open-World Video Instance Segmentation and Captioning (NeurIPS 2024 · similarity 0.87)
- OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
