Research map / Large language models / Language, safety and interpretability
PARDEN, Can You Repeat That? Defending against Jailbreaks via Repetition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PARDEN, a jailbreak detector that asks an aligned LLM to repeat its own output and flags outputs whose BLEU similarity to the repeat is low, requiring no finetuning or white-box access.
Problem
Safety-aligned LLMs such as Llama 2 and Claude 2 remain vulnerable to jailbreaks, and existing LLM-based safeguards that self-classify outputs suffer from a domain shift between the model's self-censoring alignment format and a classification format.
Approach
Instead of asking the LLM to classify content as harmful, ask it to repeat its own output; benign outputs are reproduced nearly verbatim (BLEU near 1) while harmful ones trigger a refusal (low BLEU), so a BLEU threshold separates them. This keeps the task within the self-censoring format the model was aligned on and operates purely on the output space.
Key results
- On Llama-2-7B, PARDEN reaches AUC 0.958 vs 0.922 for Classifier-Suffix, and FPR at 90% TPR falls from 24.8% to 2.0%.
- For Claude-2.1 on harmful strings, PARDEN improves (TPR, FPR) from (69.2%, 2.72%) to (90.0%, 1.09%).
- PARDEN lowers FPR at matched TPR in 9 of 10 scenarios where Llama or Claude is the defender; Mistral-7B, which is not safety-aligned, does not fit the method's premise.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace output self-classification with repetition and BLEU-based detection | Llama2-7B jailbreak detection · Harmful behaviours | AUC: 0.922 -> 0.958 | Helped |
| Replace output self-classification with repetition and BLEU-based detection | Llama2-7B jailbreak detection · Harmful behaviours | FPR at TPR 90%: 24.8% -> 2.0% | Helped |
| Use PARDEN instead of PPLX_5 | Llama2-7B jailbreak detection · Harmful behaviours | AUC: 0.660 -> 0.958 | Helped |
| Increase the repeat-token cap from 5 to 60 (ablation) | PARDEN · Harmful behaviours | AUC: 0.944 -> 0.960 | Helped |
| Repeat the full sentence instead of capping repetition at 60 tokens (ablation) | PARDEN · Harmful behaviours | AUC: 0.960 -> 0.954 | Hurt |
| Add in-context examples to the repeat prompt (ablation) | PARDEN with 5 repeat tokens · Harmful behaviours | AUC: 0.7566 -> 0.9441 | Helped |
| Add in-context examples to the repeat prompt (ablation) | PARDEN with 60 repeat tokens · Harmful behaviours | AUC: 0.9395 -> 0.9599 | Helped |
| Add in-context examples to the repeat prompt (ablation) | PARDEN with full-sentence repetition · Harmful behaviours | AUC: 0.9405 -> 0.9545 | Helped |
Surprising findings
- Mistral-7B, which is not safety-aligned, cannot be defended by PARDEN because its benign and harmful BLEU distributions do not separate.
- Repeating only the first 5 to 60 tokens performs as well as or better than repeating the full output, since long repetitions accumulate error.
Limitations
- Assumes a safety-aligned base model; relies on the model's intrinsic safeguarding mechanisms.
- False negatives on outputs that are harmless in isolation but have malicious intent, such as fake reviews.
- Evaluation uses a self-constructed dataset of 484 Llama2 and 539 Claude-2.1 jailbreaks built with prompt injection, and perplexity baseline comparison involves confounds.
Experimental setup
Datasets: open-instruct-v1 AdvBench harmful behaviours AdvBench harmful strings GCG attack prompts AutoDAN attack prompts
Benchmarks: AdvBench harmful behaviours AdvBench harmful strings GCG and AutoDAN attack mix
Metrics: AUC TPR FPR at 90% TPR ROC curves BLEU score distributions
Baselines: Classifier-Suffix PPLX_5 perplexity filter Llama-2 self-classification baseline
Builds on: GCG AutoDAN Classifier-Suffix self-classification perplexity filter BLEU
Closest papers by meaning
- Mission Impossible: A Statistical Perspective on Jailbreaking LLMs (NeurIPS 2024 · similarity 0.90)
- BackdoorAlign: Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment (NeurIPS 2024 · similarity 0.89)
- Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy (CVPR 2025 · similarity 0.89)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.89)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.89)
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.