Research map / Adversarial robustness and security / Attacks on text and graphs
T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates text-to-video jailbreaking as discrete prompt optimization, using an LLM agent (GPT-4o) with joint filter-bypass and semantic-consistency losses plus prompt-variant averaging, evaluated on Pika, Luma, Kling and Open-Sora.
Problem
Text-to-video models are vulnerable to jailbreak prompts that evade safety filters and induce unsafe video content, yet there are no systematic optimization-based attack methods for T2V models.
Approach
Treats T2V jailbreaking as a discrete token-level optimization problem, combining a filter-bypass loss (binary filter penalty plus unsafe-frame ratio) with a semantic-consistency loss (CLIP prompt similarity and CLIP similarity between a video caption and the prompt), searched iteratively by an LLM agent that proposes rewrites and scores each candidate together with K mutated variants averaged tog
Key results
- On Open-Sora, T2V-OptJail reaches 67.1% GPT-4 ASR and 68.7% human ASR, versus 55.7% and 58.7% for T2VSafetyBench.
- On Pika, T2V-OptJail reaches 55.9% GPT-4 ASR versus 47.7% for T2VSafetyBench and 14.6% for DACA.
- Removing the prompt mutation strategy lowers Open-Sora GPT-4 ASR from 67.1% to 61.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T2V-OptJail instead of T2VSafetyBench | T2V jailbreak prompts · Pika | ASR (GPT-4, %): 47.7 -> 55.9 | Helped |
| Use T2V-OptJail instead of T2VSafetyBench | T2V jailbreak prompts · Luma | ASR (GPT-4, %): 32.9 -> 43.0 | Helped |
| Use T2V-OptJail instead of T2VSafetyBench | T2V jailbreak prompts · Kling | ASR (GPT-4, %): 29.3 -> 37.9 | Helped |
| Use T2V-OptJail instead of T2VSafetyBench | T2V jailbreak prompts · Open-Sora | ASR (GPT-4, %): 55.7 -> 67.1 | Helped |
| Add Prompt Mutation (ablation) | T2V-OptJail · Pika, Luma, Kling, and Open-Sora | ASR (GPT-4, %): 51.7 to 55.9; 37.9 to 43.0; 34.4 to 37.9; 61.1 to 67.1 | Helped |
| Add Prompt Mutation (ablation) | T2V-OptJail · Open-Sora | ASR (Human, %): 62.4% to 68.7% | Helped |
| Add Prompt Mutation (ablation) | T2V-OptJail · Open-Sora | semantic similarity: 0.266 to 0.269 | Helped |
| Increase the balance factor λ (ablation) | T2V-OptJail · Open-Sora | ASR | Helped |
| Increase the balance factor λ (ablation) | T2V-OptJail · Open-Sora | semantic similarity | Hurt |
| Increase the balance factor β (ablation) | T2V-OptJail · Open-Sora | ASR | Hurt |
| Increase the balance factor β (ablation) | T2V-OptJail · Open-Sora | semantic similarity | Helped |
| Increase iterations up to 20 (ablation) | T2V-OptJail · Open-Sora | ASR | Helped |
Surprising findings
- Commercial platforms such as Kling and Luma resist the attack more than Open-Sora, with Kling at 37.9% GPT-4 ASR.
Limitations
- Requires querying generated videos during optimization, adding query budget cost.
- Open-Sora's safety filters are manually integrated and simulate real-world deployment rather than being the platform's own.
- Evaluation uses a 700-prompt subset of T2VSafetyBench due to computational costs.
Experimental setup
Datasets: T2VSafetyBench
Benchmarks: T2VSafetyBench
Metrics: attack success rate cosine similarity (CLIP)
Baselines: T2VSafetyBench DACA
Builds on: T2VSafetyBench DACA
Closest papers by meaning
- T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models (NeurIPS 2024 · similarity 0.91)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.88)
- ColJailBreak: Collaborative Generation and Editing for Jailbreaking Text-to-Image Deep Generation (NeurIPS 2024 · similarity 0.88)
- Perception-Guided Jailbreak Against Text-to-Image Models (AAAI 2025 · similarity 0.88)
- Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.88)
- JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.