Research map / Large language models / Instruction and fine-tuning
Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
Min-Seong Kim, Jongbok Won, Jeesup Park, Yonghee Choi, Dong-Jun Han
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a weight-space reparameterization for LLM fine-tuning that aims to preserve safety alignment while adapting to downstream tasks.
Problem
Fine-tuning LLMs on downstream tasks weakens previously aligned safety behavior, creating a trade-off between task adaptation and safety preservation; safety information is distributed across many dimensions in the original parameter space.
Approach
Reparameterizes model weights into a different space so that safety-relevant information is handled during downstream adaptation, instead of identifying safety neurons, layers, or update directions in the original parameter space.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reparameterize weights into a safety-conditioned basis before freezing important coefficients (ablation) | freeze-and-adapt fine-tuning · Llama-2-7B GSM8K | jailbreak ASR (%): 9.17 -> 6.90 | Improved |
| reparameterized freeze at 10% vs original-space freeze at 10% (ablation) | safety-tuned Llama-2-7B · jailbreak average | jailbreak ASR (%): 33.95 -> 14.67 | Improved |
| increase original-space freeze ratio from 10% to 50% (ablation) | full fine-tuning · Llama-2-7B | jailbreak ASR (%): 33.95 -> 18.43 | Improved |
| WSR-Tune versus full-parameter FT | Llama-2-7B-Chat · GSM8K | jailbreak ASR (%): 20.78 -> 6.90 | Improved |
| WSR-Tune versus full-parameter FT | Llama-2-13B-Chat · GSM8K | accuracy (%): 45.94 -> 49.58 | Improved |
| WSR-Tune versus full-parameter FT | Llama-3.1-8B-Instruct · MATH | accuracy (%): 12.12 -> 13.70 | Improved |
| WSR-Tune versus full-parameter FT | Llama-2-7B-Chat · GSM8K | accuracy (%): 41.17 -> 38.99 | Hurt |
| add WSR-Tune to SN-Tune | SN-Tune · Llama-2-7B-Chat GSM8K | jailbreak ASR (%): 27.18 -> 23.07 | Improved |
| add WSR-Tune to SafeInstr | SafeInstr · Llama-2-13B-Chat GSM8K | jailbreak ASR (%): 5.82 -> 2.93 | Improved |
| add WSR-Tune to SEAL | SEAL · Llama-2-13B-Chat GSM8K | jailbreak ASR (%): 10.28 -> 1.00 | Improved |
| raise freeze ratio from 10% to 20% (ablation) | WSR-Tune · Llama-2-7B-Chat GSM8K | jailbreak ASR (%): 6.90 -> 5.59 | Improved |
| vary freeze ratio 1% to 20% (ablation) | WSR-Tune · Llama-2-7B-Chat GSM8K | accuracy (%): 38.82 to 39.88 | No clear effect |
Closest papers by meaning
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.91)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.91)
- Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.90)
- Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets (ICML 2025 · similarity 0.90)
- What Makes and Breaks Safety Fine-tuning? A Mechanistic Study (NeurIPS 2024 · similarity 0.89)
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
