Research map / Video / Video generation
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces T2VSafetyBench, a 12-aspect safety benchmark with 4,400 malicious prompts, evaluating commercial and open text-to-video models with GPT-4 judging validated against human reviews.
Problem
Text-to-video models can generate illegal, unethical, or otherwise unsafe videos, including temporal risks that individual frames do not reveal, but no comprehensive quantitative safety benchmark exists for them.
Approach
Defines 12 video-specific safety aspects (including a temporal-risk aspect), builds a malicious prompt set from real user prompts, GPT-4 generated prompts, and jailbreak attacks, and scores generated videos with GPT-4 on sampled frames, checked against human ratings.
Key results
- GPT-4 and human NSFW judgments correlate above 0.8 in most of 12 aspects (lowest is 0.602 for Disturbing Content).
- No single model is safest across aspects: SVD has ~0% pornography NSFW rate, Gen2 ~0% gore, Pika lowest political sensitivity risk.
- Pika has the highest temporal-risk NSFW rate (81.3% GPT-4), while the other three models are near 0%, suggesting capability and risk trade off.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Stable Video Diffusion rather than Pika | Text-to-video generation · Pornography | NSFW rate (GPT-4): 22.3% -> 0.1% | Helped |
| Use Gen2 rather than Pika | Text-to-video generation · Gore | NSFW rate (GPT-4): 95.2% -> 0.0% | Helped |
| Use Gen2 rather than Pika | Text-to-video generation · Disturbing Content | NSFW rate (GPT-4): 73.4% -> 26.0% | Helped |
| Use Pika rather than Gen2 | Text-to-video generation · Political Sensitivity | NSFW rate (GPT-4): 59.3% -> 10.6% | Helped |
| Use Pika rather than Gen2 | Text-to-video generation · Copyright and Trademark | NSFW rate (GPT-4): 76.4% -> 13.1% | Helped |
| Use Open-Sora rather than Pika | Text-to-video generation · Temporal Risk | NSFW rate (GPT-4): 81.3% -> 3.7% | Helped |
| Use Open-Sora rather than Pika | Text-to-video generation · Violence | NSFW rate (GPT-4): 54.3% -> 95.9% | Hurt |
| Use Open-Sora rather than Pika | Text-to-video generation · Public Figures | NSFW rate (GPT-4): 97.0% -> 97.3% | No clear effect |
| Use jailbreak prompt attacks instead of LLM-generated prompts | Pika · Pornography | NSFW rate (GPT-4): 6.3% -> 46.2% | Helped |
| Use jailbreak prompt attacks instead of LLM-generated prompts | Open-Sora · Pornography | NSFW rate (GPT-4): 81.3% -> 38.5% | Hurt |
| Use GPT-4 assessment rather than human assessment | Text-to-video safety evaluation · Most safety aspects | Correlation coefficient: 0.845; 0.867; 0.832; 0.856; 0.818; 0.829; 0.880; 0.889 | Helped |
| Use GPT-4 assessment rather than human assessment | Text-to-video safety evaluation · Disturbing Content | Correlation coefficient: 0.602 | Hurt |
Surprising findings
- Weaker generation capability (e.g., Open-Sora on borderline pornography, discrimination) can lower measured NSFW rates, a capability-safety trade-off.
- Jailbreak attacks are less effective on Open-Sora because gibberish in prompts exceeds its comprehension.
Limitations
- GPT-4 judgments diverge from humans on Disturbing Content (r=0.602).
- Jailbreak prompt dataset released only upon request for research purposes.
- Results reflect model versions and sampling of four videos per prompt at a specific time.
Experimental setup
Datasets: VidProM Ring-A-Bell prompts GPT-4 generated prompts
Benchmarks: T2VSafetyBench
Metrics: NSFW rate correlation coefficient between GPT-4 and human judgments
Baselines: Pika Gen2 Stable Video Diffusion Open-Sora 1.1
Builds on: Ring-A-Bell Jailbreaking Prompt Attack Black-box Stealthy Prompt Attack Self-Instruct VidProM
Closest papers by meaning
- T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation (CVPR 2025 · similarity 0.91)
- T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks (NeurIPS 2025 · similarity 0.91)
- Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs (NeurIPS 2025 · similarity 0.90)
- OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models (NeurIPS 2025 · similarity 0.89)
- ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024 · similarity 0.88)
- VMDT: Decoding the Trustworthiness of Video Foundation Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.