Research map / Video / Video-language understanding
Video-LevelGauge: Investigating Contextual Positional Bias in Video Language Models.
Hou Xia, Zheren Fu, Fangcan Ling, Jiajun Li, Yi Tu, Zhendong Mao, Yongdong Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a commercial model instead of an open-source model | LVLMs · Video-LevelGauge MCQA | positional bias (P_ran, P_var): Gemini 2.5 Pro P_ran 2.0, P_var 0.9 vs Qwen2.5-VL-7B 12.4, 11.7 | Improved |
| enable multimodal reasoning mode (ablation) | GLM-4.5V 108B · Video-LevelGauge MCQA | P_ran: 3.4 -> 2.7 | Improved |
| enable multimodal reasoning mode (ablation) | MiMo-VL-RL 7B · Video-LevelGauge MCQA | P_var: 4.9 -> 1.8 | Improved |
| increase context length (ablation) | LVLMs (8 models) · Video-LevelGauge | positional bias | Hurt |
| use more complex contexts (long video, multi-video, interleaved video-text, long text) instead of template or short video (ablation) | LVLMs · Video-LevelGauge | positional bias | Hurt |
| scale up model size within the same series (ablation) | InternVL3, LLaVA-Video, Qwen2.5-VL, LLaVA-OV · Video-LevelGauge MCQA | P_var: LLaVA-Video 7B 3.5 -> 72B 1.2 | Improved |
| scale up Qwen2.5-VL from 7B to 32B (ablation) | Qwen2.5-VL · Video-LevelGauge open-ended | P_var: 13.8 -> 7.2 | Improved |
| increase probe length from 2 to 8 frames (ablation) | InternVL3-8B, LLaVA-OV-7B, Qwen2.5-VL-7B · Video-LevelGauge MCQA | S_meta: InternVL3-8B 59.2 -> 74.9 | Improved |
| increase probe length from 2 to 8 frames (ablation) | InternVL3-8B, LLaVA-OV-7B, Qwen2.5-VL-7B · Video-LevelGauge MCQA | positional bias (P_var): InternVL3-8B 5.8 -> 5.4 | No clear effect |
| use open-ended descriptive questions instead of MCQA (ablation) | LVLMs · Video-LevelGauge | positional bias | Hurt |
| long video training (LongVA, LongVILA) | LongVA 7B, LongVILA-1M 7B · Video-LevelGauge MCQA | P_ran: LongVA 9.2, LongVILA-1M 11.5 | No clear effect |
| use a two-stage caption-then-LLM pipeline | Caption + GPT-4 · Video-LevelGauge MCQA | P_ran: 7.4 vs Qwen2.5-VL-7B 12.4 | Improved |
Closest papers by meaning
- Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation (ICML 2026 · similarity 0.91)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.90)
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.89)
- RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video (NeurIPS 2025 · similarity 0.89)
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.88)
- Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
