Research map / Video / Video-language understanding
SafeVid: Toward Safety Aligned Video Large Multimodal Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds SafeVid-350K, a 350K-pair video safety preference dataset generated via textual video descriptions and LLM-written chosen responses, aligns VLMMs with DPO, and introduces the SafeVidBench benchmark.
Problem
Video large multimodal models fail to transfer static (text/image) safety alignment to video contexts, a phenomenon termed mismatched generalization; there is little video-specific safety preference data or benchmarking.
Approach
Use detailed textual video descriptions as an interpretive bridge so that LLMs (GPT-4) can apply text-based safety guidelines to generate video-specific chosen responses, forming a large DPO preference dataset with rejected responses from an unsafe VLMM.
Key results
- LLaVA-NeXT-Video + SafeVid-350K improves average Safety Rate on SafeVidBench-Base from 53.99% to 96.38% (+42.39) and on SafeVidBench-Challenge from 46.23% to 85.94% (+39.71).
- Qwen2.5-VL-7B + SafeVid-350K reaches 95.87% on SafeVidBench-Base and 89.78% on SafeVidBench-Challenge, gaining +18.84 and +30.00 points.
- On VLBreakBench (lower is better), LLaVA-NeXT-Video's base-set attack success rate drops from 68.00% to 16.48%; MMBench-Video reports minimal capability loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · SafeVidBench-Base | Safety Rate (Avg.): 53.99 -> 96.38; +42.39% | Helped |
| DPO-align Qwen2.5-VL-7B on SafeVid-350K | Qwen2.5-VL-7B · SafeVidBench-Base | Safety Rate (Avg.): 77.03 -> 95.87; +18.84 | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · SafeVidBench-Challenge | Safety Rate (Avg.): 46.23 -> 85.94; +39.71% | Helped |
| DPO-align Qwen2.5-VL-7B on SafeVid-350K | Qwen2.5-VL-7B · SafeVidBench-Challenge | Safety Rate (Avg.): 59.78 -> 89.78; +30.00 | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · VLBreakBench Base | Attack success rate: 68.00% to 16.48% | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · VLBreakBench Challenge | Attack success rate: 63.84% to 43.57% | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · MM-SafetyBench SD+TYPO | Safety Rate: 42.02% to 74.31% | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · HarmEval | Safety Rate: 74.73% to 94.00% | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · StrongReject | Helpful Rate: 66.67 -> 100 | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · MMBench-Video | Mean perception score: 0.975 to 1.11 | Helped |
| DPO-align LLaVA-NeXT-Video on SafeVid-350K | LLaVA-NeXT-Video · MMBench-Video | Mean reasoning score: 1.14 to 0.99 | Hurt |
| DPO-align Qwen2.5-VL on SafeVid-350K | Qwen2.5-VL · MMBench-Video | Mean perception score: 1.3125 to 1.2725 | Hurt |
Surprising findings
- Proprietary GPT-4o and GPT-4o-mini score much lower than open models on SafeVidBench-Base (55.34% and 43.97% average), while Claude-3.5-sonnet scores highest among the baselines.
Limitations
- Efficacy depends on the fidelity of textual video descriptions, so subtle visual-temporal safety cues may be missed.
- Safety taxonomy may not cover rapidly evolving misuse patterns.
- No error bars or statistical significance reported for results.
Experimental setup
Datasets: InternVid-10M-FLT SafeVid-350K MMBench-Video MM-SafetyBench VLBreakBench miniJailBreakV-28K HarmEval StrongReject
Benchmarks: SafeVidBench-Base SafeVidBench-Challenge VLBreakBench MM-SafetyBench miniJailBreakV-28K HarmEval StrongReject MMBench-Video
Metrics: Safety Rate Helpful Rate attack success rate
Baselines: Claude-3.5-sonnet GPT-4o GPT-4o-mini Gemini-2.0-flash Gemini-1.5-pro LLaVA-OneVision Qwen2.5-VL-72B InternVL2.5-78B MiniCPM-o 2.6 InternVideo2.5-Chat-8B
Builds on: DPO InternVid VideoCLIP 3H principles LLaMA-Factory
Closest papers by meaning
- Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs (NeurIPS 2025 · similarity 0.90)
- SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization (CVPR 2026 · similarity 0.90)
- SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset (NeurIPS 2024 · similarity 0.89)
- Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs (ICLR 2026 · similarity 0.88)
- GuardAlign: Robust Safety Alignment in Multimodal Large Language Models (ICLR 2026 · similarity 0.88)
- SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.