Research map / Video / Video-language understanding
AdaAlloc: Adaptive Visual Token Allocation for Long-Video Question Answering
Joungbin An, Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdaAlloc, an adaptive visual-token allocation method for long-video question answering under fixed token budgets; the abstract is truncated, so details are limited.
Problem
Long-video question answering with multimodal LLMs under strict visual-token budgets, where uniform sampling loses fine-grained local cues and caption- or similarity-based retrieval relies on lossy proxies that miss motion and state changes.
Approach
Adaptively allocates the visual-token budget across a video to balance temporal coverage and spatial detail, rather than using uniform sampling or proxy-based retrieval; exact mechanism not stated in the available text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| allocate tokens between global overview and local evidence with AdaAlloc instead of uniform sampling | Qwen3-VL-8B-Instruct · LVBench (32K) | accuracy: 46.1 -> 56.6 | Improved |
| allocate tokens with AdaAlloc instead of uniform sampling | Qwen3-VL-8B-Instruct · VideoMME w/o sub (16K) | accuracy: 67.1 -> 68.4 | Improved |
| make the global/local split question-conditioned instead of fixed Balanced (ablation) | AdaAlloc · 4 benchmarks (16K) | accuracy: LVBench 49.6 -> 51.7 | Improved |
| add a fixed global stream to a local-only variant (ablation) | AdaAlloc · 4 benchmarks (16K) | accuracy: LVBench 48.9 -> 49.6 | No clear effect |
| add temporal-memory refinement of retrieved intervals (ablation) | AdaAlloc · MLVU-Test (32K) | accuracy: 66.5 -> 67.1 | Improved |
| add overview-conditioned planning and refinement on top of question-only retrieval (ablation) | AdaAlloc · 4 benchmarks (16K) | accuracy: LVBench +2.4, MLVU-Test -0.8 | Mixed |
| use retrieval-only local clips instead of uniform sampling (ablation) | Qwen3-VL-8B-Instruct · MLVU-Test (32K) | accuracy: 61.4 -> 59.4 | Hurt |
| use five discrete policies instead of directly predicting the ratio (ablation) | AdaAlloc · 4 benchmarks (32K) | average accuracy: 64.94 -> 65.51 | Improved |
| double the planner overview density (ablation) | AdaAlloc · 4 benchmarks (16K) | accuracy: LVBench 51.7 -> 51.6 | No clear effect |
| use UniversalVTG instead of query-frame similarity retrieval (ablation) | AdaAlloc · 4 benchmarks (32K) | average accuracy: 63.85 -> 65.51 | Improved |
| apply AdaAlloc to VideoLLaMA3-7B | VideoLLaMA3-7B · 4 benchmarks (32K) | average accuracy: 56.35 -> 58.60 | Improved |
| use AdaAlloc instead of DIG/AKS frame selection with fewer tokens | Qwen3-VL-8B-Instruct · VideoMME medium (w/o sub) | accuracy: 58.4 -> 68.7 | Improved |
Limitations
- Full text unavailable; method details and results cannot be verified from the abstract.
Experimental setup
Baselines: uniform sampling caption-based retrieval frame-similarity retrieval
Closest papers by meaning
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.92)
- Not Another Text Benchmark: Putting the “Visual" Back in Visual Question Answering for Large Video Models (NeurIPS 2026 · similarity 0.91)
- Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames (NeurIPS 2025 · similarity 0.91)
- Streaming Long Video Understanding with Large Language Models (NeurIPS 2024 · similarity 0.90)
- Language-aware Visual Semantic Distillation for Video Question Answering (CVPR 2024 · similarity 0.90)
- LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
