Research map / Video / Video-language understanding
TimeTraveler: Temporal Strategy Planning with Time Dictionary for Streaming Video Understanding
Minkuk Kim, Junho Kim, Seong Tae Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TimeTraveler, a temporal strategy planning framework with a time dictionary for answering queries over streaming video by deciding how to access past, current, and future evidence.
Problem
Streaming video understanding requires answering user queries whose evidence may lie in the past, the present, or only emerge in future observations; existing systems use compact visual memories or fixed evidence-access schemes that lose fine-grained details and fail to adapt.
Approach
Plans a query-dependent temporal access strategy over a time dictionary that indexes the stream, rather than relying on compact memory or a fixed retrieval scheme. Details beyond the abstract are not stated in the provided text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TimeTraveler (Time Dictionary + temporal strategy planning) | Qwen3-VL-8B · OVO-Bench | overall accuracy (%): 51.70 -> 58.02 | Improved |
| add TimeTraveler | Qwen3-VL-8B · StreamingBench | overall accuracy (%): 59.33 -> 64.51 | Improved |
| add TimeTraveler | InternVL3.5-8B · OVO-Bench | overall accuracy (%): 47.84 -> 49.03 | Improved |
| add TimeTraveler | InternVL3.5-8B · OVO-Bench Forward Active Responding | accuracy (%): 42.59 -> 37.06 | Hurt |
| use structured eight-field dictionary instead of flat text memory (ablation) | TimeTraveler · OVO-Bench | overall accuracy (%): 55.79 -> 58.02 | Improved |
| use structured dictionary instead of visual prototype memory at equal budget (ablation) | TimeTraveler · OVO-Bench | overall accuracy (%): 53.90 -> 58.02 | Improved |
| use planner routing instead of random routing (ablation) | TimeTraveler · OVO-Bench Backward Tracing | accuracy (%): 45.05 -> 51.37 | Improved |
| use planner routing instead of always looking back to past (ablation) | TimeTraveler · OVO-Bench Backward Tracing | accuracy (%): 50.43 -> 51.37 | Improved |
| use adaptive probabilistic selection instead of fixed top-4 (ablation) | TimeTraveler past lookback · OVO-Bench Backward Tracing | accuracy (%): 49.64 -> 51.37 | Improved |
| add Explicit Text field group to schema (ablation) | Time Dictionary · StreamingBench TR | accuracy (%): 85.98 -> 90.97 | Improved |
| add Entity-Centric field group to schema (ablation) | Time Dictionary · OVO-Bench OJR | accuracy (%): 72.83 -> 74.46 | Improved |
| add future gating to Time Dictionary (ablation) | TimeTraveler · StreamingBench Proactive Output (Qwen3-VL-8B) | accuracy (%): 45.20 -> 62.90 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Dispider-7B — Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction (CVPR 2025) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- What Should a Streaming Video Model Remember? (NeurIPS 2026 · similarity 0.89)
- VITED: Video Temporal Evidence Distillation (CVPR 2025 · similarity 0.88)
- SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding (ICLR 2025 · similarity 0.88)
- StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering (NeurIPS 2026 · similarity 0.87)
- Streaming Long Video Understanding with Large Language Models (NeurIPS 2024 · similarity 0.87)
- StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
