Research map / Large language models / Language, safety and interpretability
Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks token-level and prompt-level jailbreak attacks across eight key implementation factors (target model size, fine-tuning, safety system prompt, template, attacker ability, suffix length, budget, intention) and seven defenses on AdvBench and MaliciousInstruct, releasing JailTrickBench.
Problem
Jailbreak attack evaluations are inconsistent because implementation details (target model settings, attacker settings, budgets) are rarely standardized, and attacks are seldom benchmarked against defense-enhanced LLMs.
Approach
Systematically isolates eight overlooked implementation factors (target-level and attacker-level) and then benchmarks seven representative jailbreak attacks against six defenses under unified prefix-based and GPT-4-agent-based ASR metrics, distilling the findings into a "bag of tricks".
Key results
- Model size does not strictly determine robustness; attack performance is largely independent of Llama model size.
- Safety system prompts and template choice significantly change jailbreak ASR, and fine-tuning (e.g., Vicuna vs Llama-2) substantially reduces safety alignment.
- AdvPrompter and AmpleGCG reach near 100% ASR_Prefix on undefended Vicuna-13B on AdvBench, while AutoDAN gets 92% ASR_Agent on Vicuna-13B with no defense; GPTFuzz remains strong across defenses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase target model size | Llama-family targets under AutoDAN and PAIR · Jailbreak attack settings | ASR | No clear effect |
| Fine-tune the target model on conversational or domain-specific data | LLMs, including Vicuna-13B · Jailbreak attack settings | ASR | Hurt |
| Add a safety system prompt | LLMs under token-level and prompt-level attacks · Jailbreak attack settings | ASR | Hurt |
| Use the original default template instead of the zero-shot template | LLMs under AutoDAN and PAIR · Jailbreak attack settings | ASR | Hurt |
| Use a stronger attacker LLM | Prompt-level jailbreak attacks · Jailbreak attack settings | ASR | Helped |
| Increase adversarial suffix length | GCG · Jailbreak attack settings | ASR | Helped |
| Increase attack budget | Token-level jailbreak attacks · Jailbreak attack settings | ASR | Helped |
| Increase attack budget | Prompt-level jailbreak attacks · Jailbreak attack settings | ASR | No clear effect |
| Change the attack intention | Token-level and prompt-level jailbreak attacks · HEx-PHI | ASR | Mixed |
| Apply defense methods instead of no defense | Jailbreak attacks on Vicuna-13B and LLaMA-2-7B · AdvBench and MaliciousInstruct | ASR_Agent | Mixed |
Surprising findings
- Robustness does not correlate with model size.
- Defenses such as Unlearning can increase attack success for some attacks (e.g., PAIR and TAP reach 96% ASR_Agent on Vicuna-13B under Unlearning).
Limitations
- Experiments are limited by compute and API cost (about 60 A800-80G GPUs and about $10,000 in API expenses), so fewer target models and larger datasets were tested.
- Experimental setup may not replicate real-world application scenarios.
- Prefix-based ASR is error-prone; GPT-4 agent judging is used as a complement.
Experimental setup
Datasets: AdvBench MaliciousInstruct HEx-PHI Tulu v2 SFT
Benchmarks: AdvBench MaliciousInstruct HEx-PHI
Metrics: ASR_Prefix ASR_Agent
Baselines: GCG AutoDAN AmpleGCG AdvPrompter PAIR TAP GPTFuzz
Builds on: GCG AutoDAN PAIR AdvPrompter SmoothLLM
Closest papers by meaning
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.93)
- h4rm3l: A Language for Composable Jailbreak Attack Synthesis (ICLR 2025 · similarity 0.91)
- Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.91)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.91)
- One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs (ICLR 2025 · similarity 0.91)
- JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.