Research map / Adversarial robustness and security / Attacks on text and graphs
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Robust Prompt Optimization (RPO), a minimax-objective method that optimizes a 20-token defensive suffix against adaptive jailbreaks, transferring it to GPT-4 and other models.
Problem
LLMs remain vulnerable to jailbreaking attacks, and existing defenses do not generalize across multiple attacks or account for adaptive adversaries.
Approach
Formalizes a minimax defensive objective that incorporates the adversary's worst-case jailbreak into the defense loss, and optimizes a discrete, transferable system-level suffix via GCG-style coordinate search, selecting the strongest current jailbreak (PAIR/GCG/JBC) at intervals during optimization.
Key results
- On JailbreakBench, RPO reduces ASR to 6% on GPT-4 and 0% on Llama-2 against PAIR, the strongest attack, outperforming baseline defenses.
- On HarmBench transfer to unseen attacks, RPO lowers average ASR for Vicuna-13B from 48.0% to 29.6% and for GPT-4 from 19.2% to 15.7%.
- RPO costs 20 extra tokens at inference and keeps MMLU largely unchanged while reducing MT-Bench for Vicuna-13B from 6.57 to 5.96.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the RPO defensive suffix | Vicuna-13B · JailbreakBench PAIR adaptive | ASR: 82% -> 20% | Helped |
| Add the RPO defensive suffix | Llama-2-7B · JailbreakBench PAIR adaptive | ASR: 4% -> 0% | Helped |
| Add the RPO defensive suffix | Vicuna-13B · JailbreakBench GCG adaptive | ASR: 58% -> 1% | Helped |
| Add the RPO defensive suffix | Llama-2-7B · JailbreakBench GCG adaptive | ASR: 2% -> 0% | Helped |
| Add the RPO defensive suffix | Vicuna-13B · HarmBench average | ASR: 48.0 -> 29.6 | Helped |
| Add the RPO defensive suffix | Llama-2-7B · HarmBench average | ASR: 9.7 -> 3.2 | Helped |
| Add the RPO defensive suffix | GPT-3.5 · HarmBench average | ASR: 27.2 -> 18.5 | Helped |
| Add the RPO defensive suffix | GPT-4 · HarmBench average | ASR: 19.2 -> 15.7 | Helped |
| Add the RPO defensive suffix | Vicuna-13B · MT-Bench | score: 6.57 -> 5.96 | Hurt |
| Add the RPO defensive suffix | Vicuna-13B · MMLU | score: 0.50 -> 0.49 | Hurt |
| Add the RPO defensive suffix | Llama-2-7B · MT-Bench | score: 6.18 -> 6.05 | Hurt |
| Add the RPO defensive suffix | Llama-2-7B · MMLU | score: 0.46 -> 0.46 | No clear effect |
Surprising findings
- Optimizing only on harmful instructions does not cause unnecessary refusal of benign prompts, though the suffix sometimes gets mentioned in responses to short queries.
- Full robustness is not typically achieved; on HarmBench the improvement is much smaller than on JailbreakBench, and RPO fails to break GCG under adaptive attack only for 500 steps.
Limitations
- Does not cover multimodal models, LLM agents, deception or malicious code generation.
- Gains on HarmBench transfer to unseen attacks are much smaller than on JailbreakBench; full robustness is not achieved.
- Suffix is not semantically meaningful and may cause minor degradation on short benign queries and MT-Bench.
Experimental setup
Datasets: AdvBench JailbreakBench HarmBench JailbreakChat prompts
Benchmarks: JailbreakBench HarmBench MT-Bench MMLU
Metrics: attack success rate MT-Bench score MMLU accuracy
Baselines: SmoothLLM Perplexity Filter Rephrasing Few-Shot defense GCG PAIR AutoDAN TAP PAP Jailbreak Chat
Builds on: GCG AutoPrompt adversarial training PAIR
Closest papers by meaning
- JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.92)
- Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.92)
- One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs (ICLR 2025 · similarity 0.92)
- RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction (ICLR 2025 · similarity 0.91)
- REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective (ICML 2025 · similarity 0.91)
- Guaranteed Jailbreaking Defense via Disrupt-and-Rectify Smoothing (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.