Research map / Video / Video-language understanding
From Label Priors to Task Evidence: Long-Video Frame Selection via Bayesian GRPO
Shuochen Chang, Bingjie Gao, Qingyang Liu, Qianli Ma, Yibo Miao, Haonan Zhao, Zhaohe Liao, Xiaofeng Zhang, Jiangtong Li, Li Niu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Bayesian GRPO-based frame selector for long-video MLLMs that replaces static keyframe label priors with task-evidence signals from the downstream model.
Problem
Long-video understanding with MLLMs is limited by quadratic attention and context windows; existing retrieval/frame-selection methods rely on supervised learning over static keyframe annotations, which often do not match the information the downstream MLLM needs for reasoning.
Approach
Shifts frame selection from label-prior supervision to task-evidence optimization, training the selector with a Bayesian formulation and GRPO-style group-relative reinforcement learning so that selected frames are rewarded by their usefulness to the downstream MLLM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace label reward with NLL task-evidence reward in Stage II (ablation) | Bayesian GRPO selector · Video-MME (8 frames) | accuracy: 58.3 -> 61.5 | Improved |
| add Stage II RL with label reward on top of label prior (ablation) | Label Prior selector · Video-MME (32 frames) | accuracy: 63.6 -> 63.8 | No clear effect |
| add Stage I label prior (pairwise ranking) to SigLIP2 selection (ablation) | SigLIP2 selector · Video-MME (8 frames) | accuracy: 54.7 -> 58.5 | Improved |
| add NLL-based Stage II on top of the label prior (ablation) | Label Prior selector · MLVU avg (32 frames) | accuracy: 68.2 -> 70.3 | Improved |
| use NLL dense reward instead of RLVR-style correctness reward (ablation) | Bayesian GRPO selector · Video-MME (32 frames) | accuracy: 62.1 -> 65.2 | Improved |
| use sparse correctness reward instead of label reward (ablation) | Stage II reward · LongVideoBench (8 frames) | accuracy: 57.6 -> 56.5 | Hurt |
| train Stage II on the target MLLM (InternVL-3-8B) instead of Qwen2.5-VL-7B (ablation) | selector evaluated on InternVL-3-8B · Video-MME (32 frames) | accuracy: 66.8 -> 66.5 | No clear effect |
| add NLL Stage II selector trained on frozen Qwen2.5-VL, applied to Qwen3-VL-8B | Qwen3-VL-8B with 32 frames · MLVU avg | accuracy: 65.3 -> 72.7 | Improved |
| increase frame budget K from 8 to 32 (ablation) | Bayesian GRPO selector · Video-MME (Qwen2.5-VL-7B) | accuracy: 61.5 -> 65.2 | Improved |
| increase frame budget K from 64 to 128 (ablation) | Bayesian GRPO selector · MLVU avg (Qwen2.5-VL-7B) | accuracy: 73.3 -> 73.4 | No clear effect |
| use the selector instead of uniform sampling with 8 frames | Qwen2.5-VL-7B · Video-MME overall | accuracy: 52.6 -> 61.5 | Improved |
| use the selector instead of uniform sampling at 32 frames | InternVL-3.5-8B · Video-MME overall | accuracy: 61.1 -> 64.1 | Improved |
Experimental setup
Builds on: GRPO
Closest papers by meaning
- Adaptive Keyframe Sampling for Long Video Understanding (CVPR 2025 · similarity 0.89)
- BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding (CVPR 2025 · similarity 0.89)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.89)
- Frame-Voyager: Learning to Query Frames for Video Large Language Models (ICLR 2025 · similarity 0.88)
- M-LLM Based Video Frame Selection for Efficient Video Understanding (CVPR 2025 · similarity 0.88)
- Flexible Frame Selection for Efficient Video Reasoning (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
