Research map / Large language models / Instruction and fine-tuning
No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms
Joshua Kazdan, Abhay Puri, Rylan Schaeffer, Lisa Yu, Chris Cundy, Jason Stanley, Sanmi Koyejo, Krishnamurthy Dj Dvijotham
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on NOICE refuse-then-comply data instead of affirmative-prefix data | GPT-4o · HeX-PHI | ASR: 0.086 -> 0.57 (Implicit 0.06) | Improved |
| NOICE fine-tuning on Claude-3-Haiku | Claude-3-Haiku · HeX-PHI | ASR: 0.0133 -> 0.72 | Improved |
| add Aligned Model Defense (aligned model writes first tokens) against Implicit attack (ablation) | Llama-3-8B-Instruct · HeX-PHI (5000 examples) | ASR: 0.56 -> 0.10 | Improved |
| add Aligned Model Defense against NOICE (ablation) | Llama-3-8B-Instruct · HeX-PHI (5000 examples) | ASR: 0.56 -> 0.48 | Mixed |
| add Llama-Guard output filter against NOICE (ablation) | Llama-3-8B-Instruct · HeX-PHI (5000 examples) | ASR: 0.56 -> 0.47 | Mixed |
| add Llama-Guard output filter against ISA (ablation) | Llama-3-8B-Instruct · HeX-PHI (5000 examples) | ASR: 0.73 -> 0.11 | Improved |
| constrain fine-tuning optimization on first 5 tokens (ablation) | NOICE · HeX-PHI | ASR: 0.56 -> 0.59 | No clear effect |
| constrain fine-tuning optimization on first 5 tokens (ablation) | Implicit attack · HeX-PHI | ASR: 0.56 -> 0.24 | Hurt |
| increase model size from 1B to 8B (ablation) | NOICE on Llama 3 · HeX-PHI | ASR: 0.24 -> 0.56 | Improved |
| increase model size from 2B to 27B (ablation) | NOICE on Gemma 2 · HeX-PHI | ASR: 0.32 -> 0.28 | No clear effect |
| increase fine-tuning data amount (ablation) | NOICE · HeX-PHI | ASR | Improved |
Closest papers by meaning
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.90)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.90)
- Fundamental Limitations in Pointwise Defences of LLM Finetuning APIs (NeurIPS 2025 · similarity 0.90)
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.89)
- Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs (NeurIPS 2025 · similarity 0.89)
- BackdoorAlign: Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
