Research map / Large language models / Instruction and fine-tuning
SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SaLoRA, a LoRA variant that uses a fixed safety projection module computed from ~300 harmful prompt-response pairs plus a task-specific SVD-based initialization, so benign fine-tuning preserves Llama-2-chat safety alignment.
Problem
LoRA fine-tuning, even on benign data, degrades the safety alignment of LLMs by perturbing the internal features that trigger refusal of harmful prompts.
Approach
Add a fixed projection C_S = I - U_C U_C^T (top singular vectors of the original layer's outputs on harmful prompts) that keeps the adapter's new features orthogonal to safety features, plus a task-specific SVD initialization of the trainable A and B adapters that is balanced in norm and re-parameterizes the residual weight so the initial model is unchanged.
Key results
- On Llama-2-chat-7B with rank 16 on Alpaca, SaLoRA reduces harmful rate to 3.5% versus 23.7% for LoRA (rank 16) and 31.7% for LoRA at rank 32 (SaLoRA 4.4%).
- On commonsense-15k with rank 16, SaLoRA reaches 62.9% average accuracy versus 59.2% for LoRA and 0.0% harmful rate versus 7.8% for LoRA.
- Under multi-GCG jailbreak attacks, SaLoRA's harmful rate stays near the original model's level and beats LoRA, InferAligner and Safe LoRA by a clear margin (per figure).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SaLoRA instead of LoRA | Llama-2-chat-7B, rank 16 · AdvBench test subset | harmful rate: 23.7% -> 3.5% | Helped |
| Use SaLoRA instead of LoRA | Llama-2-chat-13B, rank 16 · AdvBench test subset | harmful rate: 15.7% -> 2.9% | Helped |
| Use SaLoRA instead of LoRA | Llama-3.1-Instruct-8B, rank 16 · AdvBench test subset | harmful rate: 11.7% -> 2.9% | Helped |
| Use SaLoRA instead of LoRA | Mistral-7B-Instruct-v0.3, rank 16 · AdvBench test subset | harmful rate: 82.5% -> 31.7% | Helped |
| Add Self-Reminder to SaLoRA | Llama-2-chat-7B, rank 16 · AdvBench test subset | harmful rate: 3.5% -> 1.4% | Helped |
| Use SaLoRA instead of DoRA | Llama-2-chat-7B · Commonsense-15k, Avg. Acc., rank 16 | accuracy: 61.3 -> 62.9 | Helped |
| Use SaLoRA instead of PiSSA | Llama-2-chat-7B · Commonsense-15k, Avg. Acc., rank 64 | accuracy: 65.9 -> 65.7 | Hurt |
| Use SaLoRA instead of PiSSA | Llama-2-chat-7B · Commonsense-15k, HellaSwag, rank 16 | accuracy: 38.1 -> 49.3 | Helped |
| Replace SaLoRA's task-specific initialization with LoRA initialization (ablation) | SaLoRA · Commonsense-15k | commonsense reasoning performance | Hurt |
| Use SaLoRA instead of LoRA | Alpaca fine-tuning, rank 32 · Alpaca training | training time: 2.95 -> 2.98 | No clear effect |
Surprising findings
- On Mistral-7B-Instruct-v0.3 (47.8% baseline harmful rate), SaLoRA lowers harmful rate to 31.7%-36.5%, below the original model, which the authors attribute to strengthened safety features.
- Without the task-specific initialization, SaLoRA performs worse than vanilla LoRA on commonsense reasoning, showing the fixed safety projection hampers convergence.
Limitations
- Adapter storage is roughly twice that of vanilla LoRA since the initialization matrices must be saved.
- Relies on a small set of harmful prompt-safe response pairs (about 300) and assumes the safety features lie in a low-rank subspace, which is supported only heuristically.
- The theoretical proposition is loosely argued, and the Vaccine baseline's instability is attributed to small alignment data without deeper analysis.
Experimental setup
Datasets: Alpaca AdvBench commonsense-15k
Benchmarks: AdvBench test subset BoolQ PIQA SIQA HellaSwag WinoGrande ARC-e ARC-c OBQA
Metrics: harmful rate commonsense accuracy linear probing accuracy
Baselines: LoRA DoRA PiSSA Self-Reminder InferAligner Vaccine Safe LoRA
Builds on: LoRA PiSSA DoRA Safe LoRA Vaccine
Closest papers by meaning
- Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models (NeurIPS 2024 · similarity 0.91)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.90)
- Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (NeurIPS 2026 · similarity 0.90)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.89)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.89)
- Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.