Research map / Vision-language and multimodal / Visual question answering
T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds VHD11K, a 10K-image and 1K-video harmfulness dataset (real and generated) labeled by a three-VLM GPT-4V debate framework with in-context learning, and benchmarks existing harmfulness detectors and VLMs on it.
Problem
Existing harmful-content datasets cover few harmful object categories, use mostly real images, and lack video and synthesized content, which limits generalizable harmfulness recognition.
Approach
Formulate harmful-content annotation as a multi-agent VQA debate: a judge VLM (GPT-4V) decides after affirmative and negative debater VLMs argue, with in-context examples refining the annotator to match the harmfulness definition.
Key results
- Annotator agreement with human labels on SMID is 82.5% accuracy over 1,674 images.
- GPT-4V with long prompt achieves the best average accuracy among benchmarked methods (78.60% on VHD11K-Images, 79.60% on VHD11K-Videos).
- InstructBLIP soft-prompt-tuned on VHD11K-Images improves average accuracy on VHD11K-Images from 68.60% to 75.50% and on SMID images from 77.51% to 85.21% after tuning on VHD11K.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a definition of harmfulness to the short prompt | GPT-4V · VHD11K-Images | Avg. accuracy: 64.36 -> 78.60 | Helped |
| Add a definition of harmfulness to the short prompt | GPT-4V · VHD11K-Videos | Avg. accuracy: 71.10 -> 79.60 | Helped |
| Add a definition of harmfulness to the short prompt | InstructBLIP · VHD11K-Images | Avg. accuracy: 66.66 -> 61.84 | Hurt |
| Add a definition of harmfulness to the short prompt | InstructBLIP · VHD11K-Videos | Avg. accuracy: 67.30 -> 50.00 | Hurt |
| Add a definition of harmfulness to the short prompt | CogVLM · VHD11K-Images | Avg. accuracy: 54.85 -> 50.29 | Hurt |
| Add a definition of harmfulness to the short prompt | CogVLM · VHD11K-Videos | Avg. accuracy: 57.30 -> 52.20 | Hurt |
| Prompt-tune on VHD11K images instead of using pretrained InstructBLIP | InstructBLIP · VHD11K-Images | Avg. accuracy: 68.60 -> 75.50 | Helped |
| Prompt-tune on VHD11K videos instead of using pretrained InstructBLIP | InstructBLIP · VHD11K-Videos | Avg. accuracy: 64.00 -> 68.00 | Helped |
| Prompt-tune on VHD11K images instead of SMID | InstructBLIP · SMID Images | Avg. accuracy: 73.37 -> 85.21 | Helped |
| Prompt-tune on VHD11K images instead of SMID | InstructBLIP · VHD11K-Images | Avg. accuracy: 68.90 -> 75.50 | Helped |
| Add in-context learning samples to the annotator (ablation) | GPT-4V annotator · VHD11K-Images | F1-score: 0.64 -> 0.84 | Helped |
| Add in-context learning samples to the annotator (ablation) | GPT-4V annotator · VHD11K-Videos | F1-score: 0.77 -> 0.81 | Helped |
Surprising findings
- Fine-tuning InstructBLIP on SMID produced a strong bias toward unharmful predictions, while fine-tuning on VHD11K balanced the two classes better.
- Q16, HOD, NudeNet and Hive AI are biased toward unharmful predictions on the comprehensive dataset.
Limitations
- Annotations rely on GPT-4V as judge and debaters, so labels inherit its biases and the closed-model dependency.
- Videos are represented by only 5 uniformly sampled frames.
- Limited finetuning experiments: only InstructBLIP with soft prompt tuning was evaluated.
Experimental setup
Datasets: VHD11K SMID Pexels and YouTube videos (crawled) Google Image Search (crawled)
Benchmarks: VHD11K-Images VHD11K-Videos SMID
Metrics: accuracy harmful accuracy unharmful accuracy average accuracy multi-class accuracy TPR TNR F1-score
Baselines: Q16 HOD NudeNet Hive AI InstructBLIP CogVLM GPT-4V LLaVA-NeXT SMID
Builds on: AutoGen HarmBench Safe Latent Diffusion InstructBLIP soft prompt tuning
Closest papers by meaning
- MLLM-as-a-Judge for Image Safety without Human Labeling (CVPR 2025 · similarity 0.87)
- VLMs can Aggregate Scattered Training Patches (NeurIPS 2025 · similarity 0.86)
- Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually (AAAI 2024 · similarity 0.85)
- ELITE: Enhanced Language-Image Toxicity Evaluation for Safety (ICML 2025 · similarity 0.85)
- VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety (ICLR 2026 · similarity 0.85)
- Contamination Detection for VLMs Using Multi‑Modal Semantic Perturbations (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.