Research map / Adversarial robustness and security / Attacks on text and graphs

AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs

ICML 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains an LLM (AdvPrompter) with an alternating optimization scheme to generate fluent, human-readable adversarial suffixes that jailbreak target LLMs in seconds, and shows that fine-tuning on these suffixes improves robustness.

Problem

Automated jailbreaking of aligned LLMs either yields high-perplexity, non-human-readable adversarial suffixes or requires costly per-prompt discrete optimization, which does not scale to many attacks.

Approach

Amortize adversarial suffix search by training a conditional LLM to predict suffixes from the instruction, alternating between a graybox beam-search optimizer (AdvPrompterOpt, using only target log-probs and a base-LM perplexity penalty) and supervised regression of the AdvPrompter onto the found suffixes.

Key results

• On AdvBench, AdvPrompter with warm start reaches 89.4 ASR@10 on Vicuna-13b and 99.4 on Mistral-7b, with perplexity ~40 versus ~114k for universal GCG suffixes.
• Transfer to GPT-3.5-0301 on AdvBench gives ASR@10 of 89.0 versus 75.0 for universal GCG.
• Adversarial fine-tuning on AdvPrompter data drops Vicuna-7b validation ASR from 43.3 to 0.9 and Mistral-7b from 58.7 to 0.0 with small MMLU/MT-bench changes.

Measured effects reported in the paper

Surprising findings

• Gradient information from the TargetLLM is not critical: the graybox AdvPrompterOpt uses only output log-probabilities and still matches or beats AutoDAN.
• Llama2-7b-chat was too hard to attack and gave a weak training signal, while Vicuna-7b was too easy and produced less meaningful suffixes.

Limitations

• Keyword-based ASR can produce false positives and negatives, so main results rely on keyword matching with StrongREJECT only as a secondary check.
• Blackbox results use model transfer from Vicuna-13b and Llama2 is treated as a simulated blackbox; only some defenses (e.g., safety system messages) were validated.
• Training requires roughly 10-20 hours per target model.

Experimental setup

Datasets: AdvBench HarmBench

Benchmarks: AdvBench HarmBench MMLU MT-bench

Metrics: ASR@1 ASR@10 perplexity StrongREJECT score

Baselines: GCG AutoDAN PAIR TAP PAP BEAST PPO-based AdvPrompter variant

Builds on: AutoDAN GCG LoRA PPO

Closest papers by meaning

• Improved Generation of Adversarial Examples Against Safety-aligned LLMs (NeurIPS 2024 · similarity 0.91)
• An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.90)
• Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.90)
• GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.90)
• LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.89)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective (ICML 2025 · similarity 0.89)