Research map / Video / Video-language understanding
VEU-Bench: Towards Comprehensive Understanding of Video Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VEU-Bench, a 19-task benchmark for video editing understanding across recognition, reasoning and judging levels, plus VEU-50K training data and the Oscars expert model fine-tuned from Qwen2-VL-7B with LoRA.
Problem
Video Large Language Models are unevaluated on video editing understanding (VEU), i.e., recognizing and reasoning about editing components such as shot size, camera motion, cuts and transitions, and existing VEU benchmarks focus mostly on classification.
Approach
A multi-level VEU benchmark (recognition, reasoning, judging) with an ontology-based knowledge-base annotation pipeline that rewrites abstract editing patterns into video-specific QA, scored by an LLM judge with a pattern-matching regularizer; a LoRA-tuned Qwen2-VL-7B (Oscars) is trained on the resulting VEU-50K data.
Key results
- Oscars-7B reaches an overall Score_all of 2.54 vs 2.64 for GPT-4o and 2.44 for Gemini-1.5-Pro, and +0.72 over Qwen2-VL-7B on VEU-Bench.
- Oscars improves over the best open-source model LLaVA-OneVision by 28.3% overall on VEU-Bench, per the paper.
- Training on VEU data improves Qwen2-VL-7B on general benchmarks, e.g. +8.5 on TempCompass Order and +7.3 on VideoMME short attribute perception.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Qwen2-VL-7B on VEU-50K | Qwen2-VL-7B · VEU-Bench test | Scoreall: 1.82 -> 2.54 | Helped |
| Fine-tune Qwen2-VL-7B on VEU-50K | Qwen2-VL-7B · VEU-Bench test | Scoremc: 2.33 -> 2.85 | Helped |
| Fine-tune Qwen2-VL-7B on VEU-50K | Qwen2-VL-7B · VEU-Bench test | Scoreoe: 1.31 -> 2.23 | Helped |
| Fine-tune Qwen2-VL-7B on VEU-50K | Qwen2-VL-7B · VideoMME-short, MVBench, and TempCompass reasoning-related tasks | score: 7.3%; 4.3%; 3.7%; 6.0%; 5.5%; 3.0%; 8.5%; 7.0%; 4.2% | Helped |
| Add editing-component definitions to the prompt (ablation) | Qwen2-VL-7B · VEU-Bench | score: 1.39 -> 1.58 | Helped |
| Add general video-task guidance to the prompt (ablation) | Qwen2-VL-7B · VEU-Bench | score: 1.39 -> 1.49 | Hurt |
| Add editing-component definitions to the prompt (ablation) | VideoLLaMA2-7B · VEU-Bench | score: 1.36 -> 1.45 | Helped |
| Add general video-task guidance to the prompt (ablation) | VideoLLaMA2-7B · VEU-Bench | score: 1.36 -> 1.60 | Helped |
| Add editing-component definitions to the prompt (ablation) | Gemini-1.5-Pro · VEU-Bench | score: 2.25 -> 2.26 | No clear effect |
| Add general video-task guidance to the prompt (ablation) | Gemini-1.5-Pro · VEU-Bench | score: 2.25 -> 2.21 | Hurt |
| Include pattern-matching regularization in the scoring system (ablation) | LLM evaluator · VEU-Bench answer evaluation | Spearman coefficient: 0.67 -> 0.86 | Helped |
Surprising findings
- Some Vid-LLMs (e.g., VideoLLaMA2) score below random chance on shot speed and shot motion recognition, attributed to uniform frame sampling.
- Several open-source models score lower on judging than reasoning for cut type, and average reasoning/judging scores of current Vid-LLMs are below 2 out of 5.
Limitations
- Frame sampling is uniform, limiting perception of speed and motion in current Vid-LLMs.
- LLM-judge scoring depends on GPT-4 and the knowledge-base ontology, so reliability is tied to those components.
Experimental setup
Datasets: AVE MovieCuts AutoTransition VEU-50K
Benchmarks: VEU-Bench Video-MME MVBench TempCompass
Metrics: accuracy Score_mc Score_oe Score_all (0-5 scale) pattern-match plus information-match score
Baselines: Gemini-1.5-Pro GPT-4o InternLM-XComposer-7B InternVL2-8B LLaVA-OneVision-7B LLaVA-Video-7B Video-CCAM-7B VideoLLaMA2-7B VILA-8B Oryx-7B Qwen2-VL-7B
Builds on: Qwen2-VL VideoChatGPT LoRA
Closest papers by meaning
- VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment (AAAI 2025 · similarity 0.91)
- E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding (NeurIPS 2024 · similarity 0.88)
- Señorita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists (NeurIPS 2025 · similarity 0.88)
- EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models (AAAI 2025 · similarity 0.88)
- SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding (CVPR 2025 · similarity 0.88)
- Learning Action and Reasoning-Centric Image Editing from Videos and Simulation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.