Research map / Adversarial robustness and security / Attacks on text and graphs
REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a REINFORCE-based adaptive, distributional, semantic objective (judge-based harmfulness reward over sampled responses) plugged into GCG and PGD jailbreak attacks, improving attack success on aligned LLMs and circuit-breaker defended Llama 3.
Problem
Optimization-based jailbreak attacks maximize the likelihood of a fixed affirmative response, which is inconsistent and non-adaptive, so they can overestimate LLM robustness.
Approach
Treat the LLM as a policy over responses and maximize the expected harmfulness reward (judge probability) over sampled responses via the REINFORCE policy gradient with a leave-one-out baseline, using a seed/random/greedy/harmful sampling strategy, and plug this loss into GCG and PGD.
Key results
- REINFORCE-GCG ASR@512 on Llama 3 8B rises from 0.35 (affirmative) to 0.73; on Gemma 1.1 2B from 0.57 to 0.88.
- On Llama 3 8B with circuit breaker defense, ASR@512 goes from 0.02 (affirmative GCG) to 0.23 with REINFORCE and 0.50 when seeded with a historical successful response on the successful subset.
- REINFORCE-PGD ASR@512 on Gemma 1.1 7B rises from 0.54 to 0.84; Llama 2 7B improves only marginally (0.17 to 0.22).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the affirmative objective with REINFORCE in GCG | GCG on Gemma 1.1 2B · HarmBench standard behaviors | ASR@512: 0.57 -> 0.88 | Helped |
| Replace the affirmative objective with REINFORCE in GCG | GCG on Gemma 1.1 7B · HarmBench standard behaviors | ASR@512: 0.63 -> 0.87 | Helped |
| Replace the affirmative objective with REINFORCE in GCG | GCG on Llama 2 7B · HarmBench standard behaviors | ASR@512: 0.32 -> 0.56 | Helped |
| Replace the affirmative objective with REINFORCE in GCG | GCG on Llama 3 8B · HarmBench standard behaviors | ASR@512: 0.35 -> 0.73 | Helped |
| Replace the affirmative objective with REINFORCE in GCG | GCG on Vicuna 1.5 7B · HarmBench standard behaviors | ASR@512: 0.86 -> 0.95 | Helped |
| Replace the affirmative objective with REINFORCE in PGD | PGD on Gemma 1.1 2B · HarmBench standard behaviors | ASR@512: 0.56 -> 0.82 | Helped |
| Replace the affirmative objective with REINFORCE in PGD | PGD on Gemma 1.1 7B · HarmBench standard behaviors | ASR@512: 0.54 -> 0.84 | Helped |
| Replace the affirmative objective with REINFORCE in PGD | PGD on Llama 2 7B · HarmBench standard behaviors | ASR@512: 0.17 -> 0.22 | Helped |
| Replace the affirmative objective with REINFORCE in PGD | PGD on Llama 3 8B · HarmBench standard behaviors | ASR@512: 0.57 -> 0.69 | Helped |
| Replace the affirmative objective with REINFORCE in PGD | PGD on Vicuna 1.5 7B · HarmBench standard behaviors | ASR@512: 0.87 -> 0.94 | Helped |
| Use REINFORCE instead of the affirmative objective | GCG on Llama 3 8B + circuit breaker · HarmBench standard behaviors | ASR@512: 0.02 -> 0.23 | Helped |
| Seed REINFORCE with a harmful response found on the base model | GCG on Llama 3 8B + circuit breaker · HarmBench standard behaviors, successful-base-attack subset | ASR@512: 0.23 -> 0.50 | Helped |
Surprising findings
- Using the affirmative gradient for candidate mutation with the REINFORCE objective for selection performs nearly as well as the full REINFORCE gradient.
- Random mutations selected with the REINFORCE objective outperform vanilla GCG by a large margin.
Limitations
- Relies on an LLM-as-a-judge reward whose shortcomings propagate into the objective.
- Evaluation relies on greedy generation, which may not reflect the response distribution.
- Circuit breaker results benefit from a seed response taken from successful attacks on the base model, which is a stronger setup.
Experimental setup
Datasets: HarmBench standard behaviors
Benchmarks: HarmBench standard behaviors
Metrics: attack success rate (ASR@128) attack success rate (ASR@512)
Baselines: GCG with affirmative objective PGD with affirmative objective AdvPrefix
Builds on: GCG PGD REINFORCE RLOO AdvPrefix
Closest papers by meaning
- Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks (NeurIPS 2024 · similarity 0.91)
- Semantic Representation Attack against Aligned Large Language Models (NeurIPS 2025 · similarity 0.89)
- Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.89)
- Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling (NeurIPS 2024 · similarity 0.89)
- AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs (ICML 2025 · similarity 0.89)
- Adversarial Search Engine Optimization for Large Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.