Research map / Video / Video-language understanding
Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Video-SafetyBench, a 2,264-pair video-text safety benchmark with harmful and benign (video-referential) queries, a subject-image plus motion-text video generation pipeline, and the RJScore logit-based judge metric, used to evaluate 24 video LVLMs.
Problem
Existing multimodal safety benchmarks focus on static image inputs and do not assess the safety risks introduced by video's temporal dynamics and video-referential implicit malice in video LVLMs.
Approach
Decompose video semantics into a subject image (what is shown) and motion text (how it moves), generated by LLM/T2I/LVLM stages, to synthesize query-relevant videos paired with harmful and benign video-referential queries; score harmfulness with RJScore, the expected value of a 1-5 toxicity score from judge-token softmax probabilities, with a calibrated threshold (2.85).
Key results
- Benign-query video prompts achieve an overall average ASR of 67.2% across 24 video LVLMs, versus 39.1% for harmful queries.
- RJScore with Qwen2.5-72B and a calibrated threshold of 2.85 reaches 91% agreement with human annotations.
- Video inputs yield an average 8.6% higher ASR than static images; Qwen2.5-VL-7B/32B/72B benign ASR is 68.7%, 73.2%, and 74.0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use benign video-referential queries instead of harmful explicit queries | Video LVLMs evaluated on Video-SafetyBench · Video-SafetyBench | ASR: 39.1% -> 67.2% | Hurt |
| Scale Qwen2.5-VL from 7B to 32B | Qwen2.5-VL · Video-SafetyBench, benign queries | ASR: 68.7% -> 73.2% | Hurt |
| Scale Qwen2.5-VL from 32B to 72B | Qwen2.5-VL · Video-SafetyBench, benign queries | ASR: 73.2% -> 74.0% | Hurt |
| Use GPT-4o instead of GPT-4o-mini | GPT-4o series · Video-SafetyBench, benign queries | ASR: 35.9% -> 43.3% | Hurt |
| Increase sampled video frames from 1 to 64 | Qwen2.5-VL-72B · Video-SafetyBench, benign queries | ASR: 66.9% -> 77.2% | Hurt |
| Increase sampled video frames from 1 to 64 | LLaVA-Video-72B · Video-SafetyBench, harmful queries | ASR: 52.0% -> 72.0% | Hurt |
| Use video inputs rather than static-image inputs | Video LVLMs · Video-SafetyBench and static-image comparison | ASR: +8.6% | Hurt |
| Add a defensive system prompt | InternVL2.5-78B · Video-SafetyBench | ASR: -17.2% | Helped |
Surprising findings
- Larger models within the same series are not safer; Qwen2.5-VL ASR on benign queries increases with scale.
- Proprietary models such as Claude 3.7 Sonnet and GPT-4o show much lower benign-query ASR (23.8% and 43.3%) than most open-source models.
Limitations
- Videos are synthesized by commercial generators rather than collected from real-world footage.
- Judge threshold calibration relies on a single 1,132-pair human-labeled set and a 5-fold procedure.
- Human annotation depends on 40 participants with AI expertise, whose labels may be subjective.
Experimental setup
Datasets: BeaverTails MMFakeBench HADES MM-SafetyBench MSCOCO 2014
Benchmarks: Video-SafetyBench MM-SafetyBench FigStep HADES VLSBench JailbreakV HarmBench-mm
Metrics: attack success rate RJScore agreement with human annotations F1 false positive rate false negative rate FID VQAScore
Baselines: MM-SafetyBench FigStep HADES HarmBench-mm JailbreakV VLSBench Beaver-Dam HarmBench Llama Guard 3 GPT-4o Rule-based
Builds on: FigStep VLSBench HarmBench Llama Guard 3 BeaverTails
Closest papers by meaning
- T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models (NeurIPS 2024 · similarity 0.90)
- SafeVid: Toward Safety Aligned Video Large Multimodal Models (NeurIPS 2025 · similarity 0.90)
- Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utility (ICML 2026 · similarity 0.89)
- Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models (ICLR 2026 · similarity 0.89)
- VMDT: Decoding the Trustworthiness of Video Foundation Models (NeurIPS 2025 · similarity 0.88)
- VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.