Research map / Video / Video-language understanding
Don't Pause! Every prediction matters in a streaming video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SPOT-Bench, a multi-turn proactive streaming VideoQA benchmark scored by Timeliness-F1, and AsynKV, a training-free adaptation of Qwen2.5-VL with persistent query state and long-short term KV memory that uses dead-time for asynchronous compute.
Problem
Existing online VideoQA benchmarks are retrospective "pause-ask-play" evaluations that only score models at fixed query timestamps, so they do not measure timeliness, over-response, or under-response of streaming video models during continuous proactive monitoring.
Approach
Evaluate streaming models at every frame with a timeliness-weighted greedy slot-matching F1 (Timeliness-F1) over human-calibrated gold intervals, and adapt an offline VideoLLM training-free using a persistent per-query state tag plus long-short term KV memory that runs summarization and memory compression during dead-time.
Key results
- AsynKV achieves overall T-F1@5 of 10.2 on SPOT-Bench, versus 3.5 for offline Qwen2.5-VL with 4 frames and 1.6 for StreamingVLM.
- AsynKV reaches 79.9 on StreamingBench Real-time and 71.6 on OVO-Bench Real-time, reported as state-of-the-art over StreamBridge (77.0 and 71.3).
- Offline Qwen2.5-VL with 16 frames reaches overall Timeliness-score of 21.7 but T-F1 of only 3.0, showing over-response dominates.
Surprising findings
- Post-training for silence reduces false positives but induces unresponsiveness, and only T-F1 reveals this behavior.
- Roughly half of streaming video time (about 52% of an average SPOT-Bench video) is dead-time with no active query, and compute there does not affect response latency.
Limitations
- Interaction and Intervention subsets are small (10-21 videos per task), limiting statistical reliability.
- Semantic matching for interaction and intervention depends on GPT-5-mini as judge.
- AsynKV runs at 1 fps and has not been evaluated with audio; open-source models mostly score zero on SPG, SI and UI.
Experimental setup
Datasets: EgoExo4D HT-Step Ego4D THUMOS'15 MovieNet Ego4D Goal-Step HoloAssist EgoBlind OVO-Bench StreamingBench
Benchmarks: SPOT-Bench StreamingBench OVO-Bench
Metrics: Timeliness-F1@5 Timeliness-score accuracy GFLOPs latency
Baselines: StreamingVLM MMDuet2 Qwen2.5-VL offline GPT-5 Gemini-3-Flash VideoLLaMA3 InternVL-3.5 PerceptionLM Qwen3-VL StreamBridge StreamForest TimeChat-Online
Builds on: ReKV StreamingVLM MMDuet2 Qwen2.5-VL
Closest papers by meaning
- What Should a Streaming Video Model Remember? (NeurIPS 2026 · similarity 0.88)
- StreamForest: Efficient Online Video Understanding with Persistent Event Memory (NeurIPS 2025 · similarity 0.86)
- LiveStar: Live Streaming Assistant for Real-World Online Video Understanding (NeurIPS 2025 · similarity 0.86)
- SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding (ICLR 2025 · similarity 0.85)
- StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering (NeurIPS 2026 · similarity 0.84)
- StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.