Research map / Large language models / Instruction and fine-tuning
Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Safe LoRA, a training-free, data-free patch that projects LoRA weight updates from selected layers onto a safety subspace defined by the aligned-minus-base weight difference, to preserve safety during LoRA fine-tuning.
Problem
Fine-tuning aligned LLMs with LoRA, even on benign data, can significantly degrade their safety alignment, and existing defenses (SafeInstr, BEA) need extra safety data and are data-dependent.
Approach
Define an alignment matrix V = W_aligned - W_unaligned per layer, form a projection matrix onto its column space, and project only those layers' LoRA updates whose cosine similarity with their projection falls below a threshold (or top-K lowest), requiring no training data or training.
Key results
- On PureBad with Llama-2-7B-Chat, Safe LoRA reduces harmfulness score from 4.66 (LoRA) to 1.055 (original model 1.058) while keeping MT-Bench at 6.34 vs 6.31 for the original model.
- On Dialog Summary with Llama-2-7B-Chat, Safe LoRA lowers harmfulness from 2.63 (LoRA) to 1.297 while keeping Rouge-1 F1 near 49.79%.
- Projecting about 11% of layers on Llama-2-7B-Chat gives a good utility-safety trade-off, while Llama-3-8B-Instruct needed about 35% of layers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-2-7B-Chat · PureBad | Harmfulness score: 4.66 -> 1.055 | Helped |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-2-7B-Chat · PureBad | Utility (MT-Bench): 4.54 -> 6.34 | Helped |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-3-8B-Instruct · PureBad | Harmfulness score: 4.637 -> 1.10 | Helped |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-3-8B-Instruct · PureBad | Utility (MT-Bench): 5.85 -> 5.05 | Hurt |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-2-7B-Chat · Dialog Summary with 100 harmful examples | Harmfulness score: 2.63 -> 1.297 | Helped |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-2-7B-Chat · Dialog Summary with 100 harmful examples | Utility (Rouge-1 F1): 50.66% -> 49.79% | Hurt |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-3-8B-Instruct · Dialog Summary with 100 harmful examples | Harmfulness score: 1.65 -> 1.268 | Helped |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-3-8B-Instruct · Dialog Summary with 100 harmful examples | Utility (Rouge-1 F1): 49.37% -> 49.04% | No clear effect |
| Apply Safe LoRA projection instead of native LoRA fine-tuning | Llama-2-7B-Chat · Alpaca | Harmfulness score: 2.25 -> 1.09 | Helped |
| Use approximate C instead of exact Ĉ projection (ablation) | Safe LoRA on Llama-2-7B-Chat · PureBad | MT-Bench: 5.96 -> 6.34 | Helped |
| Use approximate C instead of exact Ĉ projection (ablation) | Safe LoRA on Llama-2-7B-Chat · PureBad | Harmfulness score: 1.18 -> 1.055 | Helped |
| Project more LoRA layers (ablation) | Safe LoRA on Llama-2-7B-Chat · Dialog Summary | Utility (Rouge F1) | Hurt |
Surprising findings
- Safe LoRA's approximate projection C outperformed the exact projection C-hat on the reported safety/utility trade-off in the Table comparison.
- On Alpaca fine-tuning with no harmful data, harmfulness rose from 1.058 to 2.25 under LoRA, showing benign fine-tuning alone degrades safety.
Limitations
- Transparency of the method may allow adaptive attacks that circumvent it.
- Utility can drop for Llama-3 when many layers are projected, and the method depends on the availability of aligned and base weights.
- Evaluation relies on keyword-based ASR and GPT-4 judging of a small set of prompts.
Experimental setup
Datasets: PureBad Dialog Summary Alpaca
Benchmarks: PureBad Dialog Summary Alpaca MT-Bench
Metrics: harmfulness score (1-5) ASR (refusal keyword-based) MT-Bench score Rouge-1 F1
Baselines: LoRA SafeInstr BEA Vaccine
Builds on: LoRA task arithmetic Vaccine
Closest papers by meaning
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.92)
- SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation (ICLR 2025 · similarity 0.91)
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.91)
- Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets (ICML 2025 · similarity 0.90)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.90)
- Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.