Research map / Large language models / Language, safety and interpretability
LLM Stinger: Jailbreaking LLMs Using RL Fine-Tuned LLMs (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes a Gemma-2B-it attacker LLM with PPO-based RL to generate adversarial suffixes that jailbreak black-box victim LLMs on HarmBench.
Problem
Automating jailbreak suffix generation against safety-trained LLMs, which existing methods do with heavy prompt engineering or white-box gradient access, is laborious and does not scale.
Approach
An attacker LLM is fine-tuned in an RL loop using binary judge feedback plus a string-similarity token-level signal, so it learns to produce new suffixes modeled on known successful universal suffixes, requiring only black-box access to victims.
Key results
- Achieves +57.2% ASR improvement over the next-best method on Llama2-7B-chat (89.3% vs 32.1% for GCG).
- Reaches 52.2% ASR on Claude 2 versus 1.9% for the next best method (+50.3%).
- Achieves 94.97% ASR on GPT-3.5 Turbo 1106 and 99.4% on Gemma-2B-it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM STINGER instead of GCG | attacks on Llama2-7B-chat · HarmBench standard behaviours test split, Llama2-7B-chat | attack success rate: 32.1 -> 89.3 | Helped |
| Use LLM STINGER instead of GCG | attacks on Vicuna-7B · HarmBench standard behaviours test split, Vicuna-7B | attack success rate: 89.9 -> 93.08 | Helped |
| Use LLM STINGER instead of PAIR | attacks on Claude 2 · HarmBench standard behaviours test split, Claude 2 | attack success rate: 1.9 -> 52.2 | Helped |
| Use LLM STINGER instead of PAIR | attacks on Claude 2.1 · HarmBench standard behaviours test split, Claude 2.1 | attack success rate: 2.5 -> 26.4 | Helped |
| Use LLM STINGER instead of TAP-T | attacks on GPT 3.5 Turbo 0613 · HarmBench standard behaviours test split, GPT 3.5 Turbo 0613 | attack success rate: 78.6 -> 88.67 | Helped |
| Use LLM STINGER instead of TAP-T | attacks on GPT 3.5 Turbo 1106 · HarmBench standard behaviours test split, GPT 3.5 Turbo 1106 | attack success rate: 60.4 -> 94.97 | Helped |
| Use LLM STINGER instead of TAP-T | attacks on GPT 4 Turbo 1106 · HarmBench standard behaviours test split, GPT 4 Turbo 1106 | attack success rate: 81.8 -> 80.50 | Hurt |
| Use LLM STINGER | attacks on Gemma-2B-it · Gemma-2B-it | attack success rate: 99.4% | Helped |
Surprising findings
- Gemma-2B-it, the attacker model itself, reaches 99.4% ASR as a victim, which is outside the HarmBench benchmark.
Limitations
- Evaluated only on a subset of HarmBench victims due to computational limits.
- Relies on a judge LLM for rewards, with manual verification only described qualitatively.
- Does not yet support token-level attack methods or multimodal models.
Experimental setup
Datasets: HarmBench standard behaviors GCG public suffixes
Benchmarks: HarmBench standard behaviors test split
Metrics: attack success rate
Baselines: GCG GCG-M GCG-T PEZ GBDA UAT AP SFS ZS PAIR TAP TAP-T
Builds on: GCG HarmBench PPO
Closest papers by meaning
- Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation (ICLR 2024 · similarity 0.89)
- Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes (NeurIPS 2024 · similarity 0.88)
- LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.88)
- GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.88)
- AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs (ICML 2025 · similarity 0.88)
- Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.