Research map / Large language models / Instruction and fine-tuning
Look Before You Leap: Enhance Attention and Vigilance Regarding Harmful Content with GuidelineLLM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GuidelineLLM, a fine-tuned guideline generator that analyzes a query's risks and feeds safety guidelines to a frozen responding LLM to reduce jailbreak attack success without safety-tuning the responder.
Problem
LLMs remain vulnerable to evolving jailbreak attacks, and existing defenses (RLHF, red-teaming, retraining) are costly in training data and compute and can degrade existing safety alignment; a generalizable, parameter-free defense is needed.
Approach
A separate fine-tuned model (GuidelineLLM) identifies risks in a query before the responding LLM answers, summarizes them into guideline suggestions that are prepended to the query, so only the guideline model is trained while the responder's weights remain unchanged. A semi-automatic loop (JailbreakLLM + GuidelineLLM + filtering by the responder) expands the T-Jailbreak and guideline datasets.
Key results
- GuidelineLLM reduces average ASR on Vicuna-7B from 63.18% (no defense) to 8.96%, and on Llama2-7B-Chat from 19.34% to 1.53%, with the paper reporting an average reduction of 34.17% ASR.
- On Llama2-7B-Chat, adding guidelines raises the false refusal rate from 1.83% to 7.31%, while helpfulness scores stay above 3.3 on a 0-4 scale.
- Fine-tuned Qwen2.5-1.5B-Instruct as GuidelineLLM performs comparably to Llama2-7B-Chat as the guideline model (e.g., average ASR 1.74% vs 1.92% on Llama2 responder).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GuidelineLLM guidelines at inference instead of using IA | Vicuna-7B · Jailbreak datasets (average) | ASR (%): 16.6 -> 8.96 | Helped |
| Add GuidelineLLM guidelines at inference instead of using IA | Llama2-7B-Chat · Jailbreak datasets (average) | ASR (%): 0.04 -> 1.53 | Hurt |
| Add GuidelineLLM guidelines at inference instead of using IA | Vicuna-13B · Jailbreak datasets (average) | ASR (%): 10.62 -> 11.63 | Hurt |
| Add GuidelineLLM guidelines at inference instead of using IA | Vicuna-7B · AutoDAN | ASR (%): 49.42 -> 17.69 | Helped |
| Add GuidelineLLM guidelines at inference instead of using IA | Llama2-7B-Chat · AutoDAN | ASR (%): 0.19 -> 0.00 | Helped |
| Remove benign guidelines from GuidelineLLM fine-tuning data (ablation) | GuidelineLLM · Jailbreak datasets (average) | ASR (%): 1.92 -> 0.02; 6.77 -> 6.47; 9.16 -> 4.96 | Helped |
| Remove benign guidelines from GuidelineLLM fine-tuning data (ablation) | GuidelineLLM · Benign queries | Helpfulness score: 3.33 -> 3.14; 3.50 -> 3.43; 3.70 -> 3.67 | Hurt |
| Remove benign guidelines from GuidelineLLM fine-tuning data (ablation) | GuidelineLLM · Benign queries | False refusal rate (FRR): 7.31 -> 7.43; 4.01 -> 5.96; 1.83 -> 2.85 | Hurt |
| Use Qwen2.5-1.5B instead of Llama2-7B-Chat as the GuidelineLLM base model (ablation) | GuidelineLLM · Jailbreak datasets (average) | ASR (%): 1.92 -> 1.74; 6.77 -> 10.47; 9.16 -> 14.63 | Mixed |
| Increase maximum guidelines from n=3 to n=7 (ablation) | GuidelineLLM with Llama2-7B-Chat responder · Jailbreak datasets (average) | ASR (%): 1.92 -> 1.52 | Helped |
| Increase maximum guidelines from n=3 to n=7 (ablation) | GuidelineLLM with Vicuna-7B responder · Jailbreak datasets (average) | ASR (%): 6.77 -> 7.93 | Hurt |
| Increase maximum guidelines from n=3 to n=7 (ablation) | GuidelineLLM with Vicuna-13B responder · Jailbreak datasets (average) | ASR (%): 9.16 -> 11.08 | Hurt |
Surprising findings
- Fine-tuning the guideline model without benign guidelines lowers ASR further but reduces helpfulness and increases false refusals.
- Many outputs judged as successful jailbreaks under GuidelineLLM were actually non-harmful (score E), indicating token-based ASR overstates attack success.
Experimental setup
Datasets: Alpaca GPTTeacher DAN SAP200 JailbreakChatGPT AutoDAN
Benchmarks: DAN SAP200 JailbreakChatGPT AutoDAN T-Jailbreak Alpaca GPTTeacher
Metrics: attack success rate harmfulness score (0-4) helpfulness score false refusal rate
Baselines: No defense Self-Reminder IA Llama3-Guard
Builds on: Self-Reminder Llama-Guard LoRA AutoDAN
Closest papers by meaning
- Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs (ICML 2025 · similarity 0.90)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.90)
- Multi-Turn Jailbreaking Large Language Models via Attention Shifting (AAAI 2025 · similarity 0.90)
- Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts? (ICLR 2025 · similarity 0.90)
- Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models (ICLR 2025 · similarity 0.89)
- Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.