Research map / Large language models / Instruction and fine-tuning

Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack

NeurIPS 2024 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Lisa, a bi-state fine-tuning defense that adds a proximal term to alternating alignment/user-data optimization to curb excess drift and reduce harmful scores in fine-tuning-as-a-service.

Problem

Harmful fine-tuning of safety-aligned LLMs: mixing a small fraction of harmful data into a user fine-tuning dataset breaks safety alignment, and fine-tuning-stage defenses are costly and unstable under asymmetric step allocation.

Approach

Alternate between optimizing an alignment dataset and the user dataset (bi-state optimization) and add a proximal term ||w - w_t||^2 to each state so iterates do not drift excessively toward the switching checkpoint, which stabilizes convergence when few steps go to alignment.

Key results

• Lisa reduces average harmful score to 39.58% vs 48.28% for SFT and 43.28% for VLGuard on Llama2-7B with 5000 samples (default setting).
• Lisa achieves average harmful score 37.53 on Mistral-7B across four tasks, versus 47.25 for SFT and 42.85 for VLGuard, with average finetune accuracy 63.74 vs 63.94 for SFT.
• Lisa with only proximal term (no alignment-data BSO) gets lowest harmful score (32.46) but its finetune accuracy drops to 87.02.

Measured effects reported in the paper

Surprising findings

• Using only the proximal term without alignment-data bi-state optimization yields the lowest harmful score but substantially lower finetune accuracy, indicating a trade-off.
• Vaccine-SFT fails to resist high harmful ratio (p=1) attacks, with Lisa remaining more robust in that setting.

Limitations

• Extra computation and memory overhead from storing switching checkpoints and proximal-term forward/backward passes.
• Weak extension to RLHF, as noted by the authors.
• Fine-tuning accuracy drops in some settings, e.g., GSM8K accuracy is lower than SFT for Opt-2.7B and with more alignment steps.

Experimental setup

Datasets: BeaverTails SST2 AGNEWS GSM8K AlpacaEval

Benchmarks: SST2 AGNEWS GSM8K AlpacaEval BeaverTails

Metrics: harmful score finetune accuracy win rate alignment loss gradient norm wall-clock time GPU memory

Baselines: NA-SFT SFT Vaccine-SFT VLGuard EWC BSO

Builds on: Vaccine VLGuard EWC LoRA

Closest papers by meaning

• Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.90)
• Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation (ICLR 2025 · similarity 0.90)
• Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack (ICML 2025 · similarity 0.89)
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.89)
• Safety Depth in Large Language Models: A Markov Chain Perspective (NeurIPS 2025 · similarity 0.88)
• Safety Layers in Aligned Large Language Models: The Key to LLM Security (ICLR 2025 · similarity 0.88)