Research map / Adversarial robustness and security / Attacks on text and graphs

JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs

AAAI 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: LLM jailbreak and injection defense · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes JailPO, a black-box jailbreak framework that trains attack models (question and template enhancers on Llama2-7B) with SFT and SimPO preference optimization, using a judge-based preference signal, to automatically generate jailbreak prompts.

Problem

Existing jailbreak attacks rely on handcrafted templates or white-box token optimization (GCG), which are hard to scale, costly in queries, and not universal across black-box LLMs.

Approach

Fine-tune LLMs to generate covert jailbreak questions and templates, then apply SimPO preference optimization using jailbreak-success scores from a judge on a surrogate aligned model (Llama2-7B), and combine the two in three attack patterns (QEPrompt, TemplatePrompt, MixAsking).

Key results

• On AdvBench with GPT-3.5, JailPO w/ TemplatePrompt reaches 15.23% ASR (ClassJudge) with a single query, versus 9.96% for Jailbroken and 2.60% for GCG.
• MixAsking achieves the highest QSR on GPT-3.5 at 36.15% (ClassJudge) after three query iterations, compared with 27.31% for Jailbroken.
• Against LLM-Guard defense, QEPrompt attains near 100% DPR on Llama2 and Mistral, markedly above baselines.

Measured effects reported in the paper

Surprising findings

• Perplexity defense reduces DeepInception and SelfCipher attack success to 0%, while covert question transformations bypass defenses more easily.
• Stronger alignment in the preference-querying model (Llama2) yields better jailbreak learning, and alignment appears to deteriorate with larger model size (Llama3-70b vs Llama2-7B).

Limitations

• Judge-based success signals (ClassJudge, refusal patterns) may be imperfect proxies for harmful content.
• Template-based attacks show reduced effectiveness on Vicuna due to semantic complexity causing irrelevant answers.
• Evaluation limited to AdvBench and a single benchmark dataset.

Experimental setup

Datasets: AdvBench GPT-3.5-augmented question and template sets

Benchmarks: AdvBench

Metrics: ASR QSR DPR high-risk response rate

Baselines: TemplateJailbreak DeepInception SelfCipher Jailbroken GCG

Builds on: SimPO GCG GPTFuzz Bradley-Terry model

Closest papers by meaning

• AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models (ICLR 2024 · similarity 0.93)
• Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.92)
• Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks (NeurIPS 2024 · similarity 0.92)
• One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs (ICLR 2025 · similarity 0.92)
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.92)
• h4rm3l: A Language for Composable Jailbreak Attack Synthesis (ICLR 2025 · similarity 0.91)