Research map / Video / Video-language understanding
StreamRAG: Enhancing Real-Time Video Understanding with Retrieval Augmentation
Junlin Xie, Quanlong Zheng, Ruifei Zhang, Kuo Wang, Yanhao Zhang, Jinguo Luo, Haonan Lu, Xiang Wan, Guanbin Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Knowledge Extraction Accelerator (retrieval knowledge base) (ablation) | Qwen2-VL-7B · OVO-Bench | R-Avg: 55.98 -> 65.33 | Improved |
| add Streaming Event Segmentation on top of KEA (ablation) | StreamRAG · OVO-Bench | R-Avg: 65.33 -> 68.54 | Improved |
| add Dynamic Knowledge Integration on top of KEA+SES (ablation) | StreamRAG · OVO-Bench | B-Avg: 51.28 -> 53.08 | Improved |
| reuse about 18% of caption tokens (ablation) | Knowledge Extraction Accelerator · OVO-Bench | latency: 15.24s -> 11.12s | Improved |
| increase token reuse from about 18% to about 33% (ablation) | Knowledge Extraction Accelerator · OVO-Bench | B-Avg: 53.08 -> 50.47 | Hurt |
| increase token reuse from about 18% to about 33% (ablation) | Knowledge Extraction Accelerator · OVO-Bench | latency: 11.12s -> 8.43s | Improved |
| use event-triggered segmentation instead of fixed-interval updates (ablation) | StreamRAG · OVO-Bench | B-Avg: 50.04 -> 53.08 | Improved |
| use more frames for fixed-interval updating (16 to 32) (ablation) | Fixed-Interval baseline · OVO-Bench | R-Avg: 65.33 -> 68.98 | Improved |
| adaptive retrieval scope instead of latest-only caption (ablation) | StreamRAG · OVO-Bench | B-Avg: 51.25 -> 53.08 | Improved |
| adaptive retrieval scope instead of using the full knowledge base (ablation) | StreamRAG · OVO-Bench | B-Avg: 51.28 -> 53.08 | Improved |
| add offline VideoRAG to ViSpeak | ViSpeak · OVO-Bench Real-Time | avg accuracy: 66.28 -> 57.21 | Hurt |
| add StreamChat-style RAG with uniform sampling to ViSpeak | ViSpeak · OVO-Bench Real-Time | avg accuracy: 66.28 -> 64.48 | Hurt |
Closest papers by meaning
- AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding (NeurIPS 2025 · similarity 0.90)
- Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding (NeurIPS 2025 · similarity 0.89)
- SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding (ICLR 2025 · similarity 0.89)
- StreamForest: Efficient Online Video Understanding with Persistent Event Memory (NeurIPS 2025 · similarity 0.88)
- Streaming Long Video Understanding with Large Language Models (NeurIPS 2024 · similarity 0.88)
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
