Research map / Video / Video-language understanding
AdShot: Benchmarking Multimodal Large Language Models for Video Advertisement Clipping
Wen Xie, Om Rastogi, Sai S Rangoju, Gijs Overgoor, Yakov Bart, Sarah Ostadabbas
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AdShot, a benchmark evaluating multimodal large language models on shot selection for video ad clipping.
Problem
Advertisers must create multiple duration variants of video ads, a labor-intensive process; the ability of MLLMs to perform this specialized editorial task is unexplored.
Approach
A benchmark framing ad clipping as shot selection so MLLMs can be evaluated on this editorial task, rather than general video understanding.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio track to the model input (ablation) | Qwen2.5-Omni 7B · AdShot (823 ad pairs, shot selection) | F1: 0.585 -> 0.668 | Improved |
| add audio track to the model input (ablation) | MiniCPM-o 2.6 8B · AdShot (823 ad pairs, shot selection) | F1: 0.644 -> 0.669 | Improved |
| add audio track to the model input (ablation) | Qwen2.5-Omni 7B · AdShot (823 ad pairs, shot selection) | MDE (s): 7.75 -> 11.18 | Hurt |
| add audio track to the model input (ablation) | MiniCPM-o 2.6 8B · AdShot (823 ad pairs, shot selection) | Precision: 0.588 -> 0.581 | No clear effect |
| scale model up within family | Qwen3-VL · AdShot (823 ad pairs, shot selection) | F1: 0.576 -> 0.611 | Improved |
| scale model up (7B to 32B) / newer version | Qwen family · AdShot (823 ad pairs, shot selection) | F1: Qwen3-VL 8B 0.576 vs Qwen2.5-VL 7B 0.582 | Mixed |
| use larger model | LLaVA-OneVision 72B vs smaller MLLMs · AdShot (823 ad pairs, shot selection) | F1: 0.464 (below random baseline 0.503) | Hurt |
| compare MLLMs against random heuristic | 15 MLLMs · AdShot (823 ad pairs, shot selection) | F1: 0.503 baseline; nine models above, six below | Mixed |
| target ads with 6-10 shots versus fewer than 6 | video-only MLLMs · AdShot (823 ad pairs, shot selection) | F1 | Improved |
| use high-recall model (Aria) instead of balanced model | Aria 25.3B · AdShot (823 ad pairs, shot selection) | MDE (s): 3.71 (Qwen3-VL 32B) -> 11.95 | Hurt |
Experimental setup
Benchmarks: AdShot
Closest papers by meaning
- EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models (AAAI 2025 · similarity 0.86)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.85)
- GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction (ICML 2026 · similarity 0.85)
- BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards (AAAI 2026 · similarity 0.85)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.85)
- Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
