Research map / Vision-language and multimodal / Multimodal LLMs
A Multimodal Benchmark for Framing of Oil & Gas Advertising and Potential Greenwashing Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces an expert-annotated multimodal (video) benchmark of 706 oil & gas ads from Facebook and YouTube with 13 framing labels, and benchmarks six VLMs in zero-shot and 1-shot settings, with entity-aware retrieval for example selection.
Problem
Detecting greenwashing-related framing in oil and gas video advertising requires multimodal analysis, but existing framing benchmarks are text-only and no VLM benchmark for video-based framing exists.
Approach
A two-domain video framing benchmark (distantly labeled Facebook climate-obstruction ads and human-annotated YouTube corporate videos) evaluated with VLMs under zero-shot and 1-shot prompting, plus an entity-restricted CLIP-based retrieval scheme for selecting the in-context example.
Key results
- GPT-4.1 1-shot reaches 69.3% micro-F1 on YouTube and 72.6% on Facebook, the best overall results.
- Best model reaches only 41.6-46.9% F1 on Green Innovation, the hardest YouTube label, while GPT-4.1 reaches 79.4% on Environment.
- Removing transcripts drops Qwen2.5-VL 32B 1-shot from 66.2% to 61.2% on YouTube and 70.5% to 60.6% on Facebook.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 1-shot prompting instead of zero-shot prompting | Qwen2.5-VL 32B · YouTube | F-score (micro): 60.7 -> 66.2 | Helped |
| Use 1-shot prompting instead of zero-shot prompting | GPT-4.1 · YouTube | F-score (micro): 71.0 -> 69.3 | Hurt |
| Use 1-shot prompting instead of zero-shot prompting | GPT-4.1 · Facebook | F-score (micro): 61.1 -> 72.6 | Helped |
| Add transcript input (ablation) | DeepSeekVL2 4.5B · YouTube and Facebook | F-score: 53.5 -> 49.7; 63.8 -> 62.3 | Mixed |
| Add transcript input (ablation) | InternVL2 8B · YouTube and Facebook | F-score: 58.3 -> 56.7; 48.4 -> 46.2 | Hurt |
| Add transcript input (ablation) | Qwen2.5-VL 7B · YouTube and Facebook | F-score: 60.0 -> 59.2; 53.6 -> 58.2 | Mixed |
| Add transcript input (ablation) | Qwen2.5-VL 32B · YouTube and Facebook | F-score: 61.2 -> 66.2; 60.6 -> 70.5 | Helped |
| Add embedding-based search for the 1-shot example (ablation) | DeepSeekVL2 4.5B · YouTube and Facebook | F-score: 46.9 -> 49.7; 43.8 -> 62.3 | Helped |
| Add embedding-based search for the 1-shot example (ablation) | InternVL2 8B · YouTube and Facebook | F-score: 52.1 -> 56.7; 39.6 -> 46.2 | Helped |
| Add embedding-based search for the 1-shot example (ablation) | Qwen2.5-VL 7B · YouTube and Facebook | F-score: 54.3 -> 59.2; 48.2 -> 58.2 | Helped |
| Add embedding-based search for the 1-shot example (ablation) | Qwen2.5-VL 32B · YouTube and Facebook | F-score: 64.0 -> 66.2; 59.1 -> 70.5 | Helped |
| Add entity restriction when selecting the 1-shot example (ablation) | DeepSeekVL2 4.5B · YouTube and Facebook | F-score: 50.5 -> 49.7; 60.6 -> 62.3 | Mixed |
Surprising findings
- GPT-4.1 performed better in zero-shot than 1-shot on YouTube (71.0 vs 69.3), unlike open-weight models.
- Smaller DeepSeek-VL2 (4.5B) matched GPT-4o-mini in 1-shot on Facebook.
Limitations
- Facebook labels are distant, derived from ad text rather than video content (20-video check gave 83% F1).
- YouTube inter-annotator agreement is modest (Fleiss' kappa 0.61; per-label average 0.46) and the dataset is small, limiting training use.
- Dataset is English-centric and biased toward large multinational O&G companies; only single runs reported.
Experimental setup
Datasets: Facebook climate obstruction ads (Holder et al.) YouTube corporate videos (new)
Benchmarks: multimodal-oil-gas-benchmark (YouTube) multimodal-oil-gas-benchmark (Facebook)
Metrics: F1 score micro-averaged F-score
Baselines: DeepSeek-VL2 InternVL2 Qwen2.5-VL GPT-4o-mini GPT-4.1
Builds on: Holder et al. climate obstruction framing dataset Rowlands et al. multi-label framing task CLIP retrieval-based prompting Whisper-1
Closest papers by meaning
- GeoReward: Mitigating Contextual Variable Overestimation in Vision-Language Models for Cross-Market Preference Prediction (ICML 2026 · similarity 0.81)
- A3: Towards Advertising Aesthetic Assessment (CVPR 2026 · similarity 0.81)
- InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma (ICLR 2025 · similarity 0.79)
- ViMU: Benchmarking Video Metaphorical Understanding (NeurIPS 2026 · similarity 0.79)
- VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR (NeurIPS 2025 · similarity 0.79)
- Uncertain Multimodal Intention and Emotion Understanding in the Wild (CVPR 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.