Research map / Large language models / Language, safety and interpretability

Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Jailbreak attacks and defenses · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ADC, a white-box token-level jailbreak that relaxes discrete suffix optimization to the continuous simplex and adaptively sparsifies vectors (sparsity set by exp of mispredicted target tokens), reaching higher ASR and efficiency than GCG on AdvBench and HarmBench.

Problem

Token-level jailbreaks such as GCG need huge query budgets because discrete token optimization is hard; naive projection of relaxed continuous optimized vectors back to one-hot tokens causes large loss increases.

Approach

Relax one-hot token choices to the probability simplex, optimize with a momentum optimizer without reparameterization, and progressively enforce sparsity S = exp(number of mispredicted target tokens) so vectors approach one-hot before projection, reducing the projection gap.

Key results

• On AdvBench behaviors, ADC reaches 96.2% ASR on Llama2-chat-7B versus 53.8% for GCG at 0.93x computing budget and about half wall-clock time.
• On HarmBench standard behaviors, ADC+ achieves the highest ASR on seven of eight LLMs (e.g., 92% on Llama2-7B-chat vs 34.5% GCG; 100% on Zephyr-beta-7B).
• On the adversarially trained Zephyr R2D2, ADC reaches 26.5% ASR while GCG gets 0.0%, though TAP (76.5%) and PAIR (57.5%) do better.

Measured effects reported in the paper

Surprising findings

• Constant sparsity of 3 yields 0.0% ASR on Llama2-chat-7B while 2 yields 62.3%, showing that no fixed sparsity works across models.
• Prompt-level methods like TAP outperform the token-level ADC on the adversarially trained Zephyr R2D2.

Limitations

• Requires white-box gradient access, so it does not apply directly to closed-source models.
• Lower ASR than prompt-level TAP on the adversarially trained Zephyr R2D2 model.
• On AdvBench strings, ADC alone underperforms GCG on Llama2-chat-7B (32.6% vs 41.3% EM).

Experimental setup

Datasets: AdvBench harmful behaviors AdvBench harmful strings HarmBench standard behaviors

Benchmarks: AdvBench harmful behaviors AdvBench harmful strings HarmBench standard behaviors

Metrics: attack success rate exact match computing budget wall-clock time early stop rate

Baselines: GCG AutoPrompt PAIR TAP AutoDan

Builds on: GCG HarmBench AdvBench

Closest papers by meaning

• Improved Techniques for Optimization-Based Jailbreaking on Large Language Models (ICLR 2025 · similarity 0.92)
• Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs (NeurIPS 2024 · similarity 0.90)
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (ICLR 2025 · similarity 0.90)
• An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks (ICML 2025 · similarity 0.90)
• AdvPrefix: An Objective for Nuanced LLM Jailbreaks (NeurIPS 2025 · similarity 0.89)
• JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.89)