Research map / Adversarial robustness and security / Black-box and transfer attacks

Improved Generation of Adversarial Examples Against Safety-aligned LLMs

NeurIPS 2024 · Adversarial robustness and security › Black-box and transfer attacks · approach: Robustness benchmarks and defenses · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts transfer-attack techniques (Skip Gradient Method and Intermediate Level Attack) to GCG-style gradient-based adversarial suffix generation against safety-aligned LLMs, raising match rate and attack success rate without extra compute.

Problem

Gradient-based discrete optimization for jailbreak adversarial suffixes is limited because input gradients w.r.t. one-hot tokens poorly predict the loss change from token replacements, leaving low white-box attack success on safety-aligned models such as Llama-2-Chat.

Approach

Reframes the one-hot gradient gap as analogous to the substitute-to-victim gap in transfer attacks, then imports SGM (down-weighting residual-module gradients, giving LSGM) and a token-level ILA variant (LILA, replacing the gradient at the first target token with a directional guide) into GCG, and combines them.

Key results

• GCG-LSGM-LILA reaches 87% match rate and 68% ASR for query-specific suffixes on Llama-2-7B-Chat, versus 54% MR and 38% ASR for GCG.
• Universal suffix AASR on AdvBench for Llama-2-7B-Chat rises from 26.68% (GCG) to 60.32% (GCG-LSGM-ILA).
• Transfer to GPT-3.5-Turbo improves average ASR from 38.30% (GCG) to 45.20% (GCG-LSGM-LILA).

Measured effects reported in the paper

Surprising findings

• Directly applying ILA to GCG deteriorates performance (44% MR vs 54% for GCG on Llama-2-7B-Chat), so ILA must be adapted to specific token representations.
• Skip connections have much greater causal effect on adversarial loss than residual modules, and residual and skip gradients are negatively correlated in most blocks.

Limitations

• Attack success is judged by a model-based HarmBench evaluator rather than perfect human judgment.
• Method is evaluated mainly on Llama-2-Chat models and a few others; generalization to other models is limited in scope.
• Gains rely on hyperparameters such as the SGM decay factor and LILA layer choice selected heuristically.

Experimental setup

Datasets: AdvBench HarmBench

Benchmarks: AdvBench HarmBench

Metrics: match rate attack success rate average ASR worst ASR best ASR time cost

Baselines: GCG AutoPrompt GCG* ILA

Builds on: Greedy Coordinate Gradient Skip Gradient Method Intermediate Level Attack AutoPrompt HarmBench

Closest papers by meaning

• AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs (ICML 2025 · similarity 0.91)
• Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidence (NeurIPS 2025 · similarity 0.91)
• Query-Based Adversarial Prompt Generation (NeurIPS 2024 · similarity 0.90)
• Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.90)
• Visual Adversarial Examples Jailbreak Aligned Large Language Models (AAAI 2024 · similarity 0.90)
• Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation (ICLR 2024 · similarity 0.90)