Research map / Large language models / Instruction and fine-tuning
Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study
Kaustubh Ponkshe, Shaan Shah, Raghav Singhal, Praneeth Vepakomma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use top-K alignment directions instead of random-K for parallel projection (ablation) | Parallel projection, Qwen-2.5 1.5B · GSM8K (fine-tuned on Useful) | accuracy: 0.53 -> 0.55 | Improved |
| use top-K alignment directions instead of random for parallel projection (ablation) | Parallel projection, Llama-3.2 1B · GSM8K (fine-tuned on Useful) | accuracy: 0.22 -> 0.25 | Improved |
| use top-K directions instead of random-K for parallel projection on harmful FT (ablation) | Parallel projection, Llama-3.2 1B · AdvBench (fine-tuned on Harmful) | harmful score: 3.37 -> 3.51 | Hurt |
| use top-K directions instead of random-K for parallel projection on harmful FT (ablation) | Parallel projection, Qwen-2.5 1.5B · AdvBench (fine-tuned on Harmful) | harmful score: 1.78 -> 1.92 | Hurt |
| increase SVD fraction from 0.01 to 0.99 in top-K parallel projection | Top-K parallel projection, Qwen-2.5 1.5B · GSM8K (fine-tuned on Useful) | accuracy: 0.50 -> 0.58 | Improved |
| increase SVD fraction from 0.01 to 0.99 in top-K parallel projection | Top-K parallel projection, Qwen-2.5 1.5B · AdvBench (fine-tuned on Harmful) | harmful score: 1.62 -> 1.97 | Hurt |
| orthogonal projection removing top-K alignment directions instead of random-K (ablation) | Contaminated FT, Qwen-2.5 7B · AdvBench harmful score | harmful score: 1.46 -> 1.60 | No clear effect |
| orthogonal projection removing top-K instead of random-K (ablation) | Contaminated FT, Qwen-2.5 7B · GSM8K | accuracy: 0.76 -> 0.75 | No clear effect |
| train safety via DPO instead of SFT on BeaverTails (ablation) | Llama-3.2 1B weight-space analyses · projection and MSO analyses | utility and harmfulness trends | No clear effect |
| restrict projection to a single layer instead of all layers (ablation) | Qwen-2.5 1.5B · GSM8K and AdvBench | utility and harmfulness | No clear effect |
| apply projections across settings (general ability check) (ablation) | Qwen-2.5 1.5B and 3B · MMLU | accuracy: 61-62% and 64-65% | No clear effect |
| apply projections (over-refusal check) (ablation) | Qwen-2.5 1.5B and 3B · XSTest | refusal rate: 3-4% and 5-6% | No clear effect |
Closest papers by meaning
- Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.91)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.90)
- The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions (ICML 2025 · similarity 0.89)
- Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.89)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.89)
- Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
