Research map / Video / Video-language understanding
AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaVideoRAG, which routes each long-video query by difficulty (no retrieval, naive multimodal retrieval, or graph retrieval) over caption/ASR/OCR/visual indexes, and releases the HiVU long-video benchmark.
Problem
Fixed-paradigm RAG for long-video understanding wastes computation on simple queries and loses information on multi-hop queries, so a single retrieval path is suboptimal for accuracy-efficiency trade-offs.
Approach
A lightweight LLM intent classifier assigns each query one of three difficulty levels and routes it to no retrieval, naive retrieval over clip-caption/ASR/OCR/visual databases, or LightRAG-style graph retrieval, with results integrated into an MLLM prompt.
Key results
- On Video-MME, Qwen2.5-VL-7B + AdaVideoRAG reaches 59.9 overall vs 47.2 base and 55.0 for VideoRAG; VideoLLaMA3 + AdaVideoRAG reaches 68.5 vs 64.2 base.
- On MLVU test, AdaVideoRAG lifts Qwen2.5-VL-7B average from 29.0 to 40.5 (+39.8%) and VideoLLaMA3-7B from 47.7 to 53.2.
- On HiVU Level-3 win-rate judged by DeepSeek-32B, VideoLLaMA3 + AdaVideoRAG wins 77.13% vs 22.87% for the base model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive retrieval augmentation | Video-LLaVA 7B · MLVU_test | AVG: 29.4 -> 37.9 | Helped |
| add adaptive retrieval augmentation | Qwen2.5-VL 7B · MLVU_test | AVG: 29.0 -> 40.5 | Helped |
| add adaptive retrieval augmentation | Qwen2.5-VL 72B · MLVU_test | AVG: 41.7 -> 45.1 | Helped |
| add adaptive retrieval augmentation | VideoLLaMA3 7B · MLVU_test | AVG: 47.7 -> 53.2 | Helped |
| use AdaVideoRAG instead of VideoRAG | Qwen2.5-VL 7B · Video-MME Overall | Overall: 55.0 -> 59.9 | Helped |
| use AdaVideoRAG instead of VideoRAG | VideoLLaMA3 7B · Video-MME Overall | Overall: 67.3 -> 68.5 | Helped |
| add adaptive retrieval augmentation | VideoLLaMA3 · HiVU | Overall Winner: 30.58% -> 69.42% | Helped |
| use AdaVideoRAG instead of VideoRAG | VideoLLaMA3 · HiVU | Overall Winner: 44.1% -> 55.9% | Helped |
| remove graph entity and relationship retrieval (ablation) | VideoLLaMA3 + AdaVideoRAG · HiVU | Overall Winner: 54.18% -> 45.82% | Hurt |
| remove vision-based feature retrieval (ablation) | VideoLLaMA3 + AdaVideoRAG · HiVU | Overall Winner: 51.77% -> 48.23% | Hurt |
| remove caption, OCR, and ASR retrieval (ablation) | VideoLLaMA3 + AdaVideoRAG · HiVU | Overall Winner: 68.75% -> 31.25% | Hurt |
| route every query through a single Level-1 path instead of adaptive routing (ablation) | AdaVideoRAG · VideoMME | Overall Score: 68.5 -> 64.2 | Hurt |
Surprising findings
- Forcing all queries to Level-3 graph retrieval (67.1) scored below the adaptive router (68.5) on Video-MME, and Level-2 forcing (67.5) also underperformed adaptive routing.
- The Qwen2.5-7B intent classifier achieved only 0.81 precision on HiVU, while Qwen2.5-1.5B reached 0.41.
Limitations
- Only models up to 32B parameters were evaluated due to limited compute.
- Only three routing levels were implemented; finer-grained classification is not explored.
- The HiVU benchmark is small (120 videos, 60 hours) and LLM-judge win rates rely on a single judge.
Experimental setup
Datasets: HiVU Video-MME MLVU
Benchmarks: HiVU Video-MME MLVU
Metrics: accuracy win rate time cost
Baselines: VideoRAG Video-LLaVA Qwen2.5-VL VideoLLaMA3 GPT-4o
Builds on: VideoRAG LightRAG Adaptive-RAG ImageBind MiniCPM-V
Closest papers by meaning
- Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understanding (NeurIPS 2025 · similarity 0.93)
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.92)
- Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames (NeurIPS 2025 · similarity 0.89)
- SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis (CVPR 2025 · similarity 0.89)
- Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding (NeurIPS 2025 · similarity 0.89)
- AdaCM^2: On Understanding Extremely Long-Term Video with Adaptive Cross-Modality Memory Reduction (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.