Research map / Large language models / Language, safety and interpretability

PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling

ICML 2025 · Large language models › Language, safety and interpretability · approach: Jailbreak attacks and defenses · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PANDAS, a hybrid many-shot jailbreaking method that adds positive affirmation phrases, negative demonstrations (refusal plus correction), and Bayesian-optimized topic-adaptive sampling of malicious demonstrations, and releases the ManyHarm dataset.

Problem

Many-shot jailbreaking (MSJ) raises attack success on safety-aligned LLMs by prepending hundreds of fabricated harmful Q&A turns, but its effectiveness can be further improved and its mechanisms are not well understood.

Approach

Modify MSJ's fabricated dialogue with three changes: insert affirmation phrases between turns, insert refusal-plus-correction phrases into existing demonstrations (negative demonstrations), and sample demonstrations from a topic-dependent distribution found by Bayesian optimization over the black-box jailbreak success rate.

Key results

• On Llama-3.1-8B (AdvBench50), PANDAS reaches 98% ASR-L at 128 shots versus 84% for MSJ at 64 shots.
• On GLM-4-9B HarmBench, PANDAS achieves 70.50% ASR-L at 32 shots versus 47.50% for MSJ.
• On OLMo-2-7B AdvBench, PANDAS reaches 57.88% ASR-L at 32 shots versus 31.35% for MSJ.

Measured effects reported in the paper

Surprising findings

• Jailbreaking effectiveness does not consistently increase with more shots; peaks often occur at 32 to 128 shots.
• Successful MSJ prompts remain effective after shuffling demonstration order, especially on Llama-3.1-8B.

Limitations

• No evaluation on proprietary models.
• Relies on malicious demonstrations from ManyHarm, which are hard to generate without uncensored models.
• Evaluation focuses on 7-9B open models, so results may not transfer to larger models.

Experimental setup

Datasets: ManyHarm AdvBench HarmBench AdvBench50

Benchmarks: AdvBench50 AdvBench HarmBench

Metrics: ASR-L ASR-R

Baselines: MSJ i-MSJ Self-Reminder ICD PPL filter Retokenization SmoothLLM

Builds on: many-shot jailbreaking i-FSJ in-context learning SmoothLLM

Closest papers by meaning

• JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.89)
• Multi-Turn Jailbreaking Large Language Models via Attention Shifting (AAAI 2025 · similarity 0.89)
• Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs (NeurIPS 2025 · similarity 0.89)
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.88)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs (ICML 2025 · similarity 0.88)
• Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy (CVPR 2025 · similarity 0.88)