Research map / Large language models / Instruction and fine-tuning

Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation

ICLR 2025 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Booster, an alignment-stage defense against harmful fine-tuning that adds a regularizer penalizing the harmful-loss reduction after a simulated normalized harmful gradient step, reducing harmful scores while preserving downstream accuracy.

Problem

Harmful fine-tuning attacks on fine-tuning-as-a-service LLMs erode safety alignment after a few harmful examples are mixed into user data; existing alignment-stage defenses leave substantial harmful scores.

Approach

Identify "harmful perturbation" (a gradient step on harmful data) as the cause of alignment breakdown, and during alignment add a regularizer that minimizes the harmful loss reduction after a simulated normalized harmful step, approximated with a first-order (second-order-term-free) three-pass update.

Key results

• Booster reduces average harmful score to 10.94 vs 33.58 for SFT across harmful ratios 0-0.2 on Llama2-7B, with average finetune accuracy 93.03 vs 90.39.
• Across Llama2-7B, Gemma2-9B, and Qwen2-7B, Booster averages harmful score 7.03 vs 41.17 for SFT with finetune accuracy 94.19 vs 94.15.
• Booster roughly triples alignment clock time relative to SFT (1.86h vs 0.54h) and adds about 8.5GB GPU memory, with no extra cost during fine-tuning.

Measured effects reported in the paper

Surprising findings

• Booster's harmful training loss at step 0 is lower than SFT's, traced to a much smaller increase of harmful loss during alignment (1.582 to 1.736 vs 1.582 to 3.920 for SFT).
• A TAR-style negative-harmful-loss tamper-resistance term made training unstable and collapse the model, motivating the reduction-rate formulation.

Limitations

• Harmful score rises with higher harmful ratio, a known weakness of alignment-stage defenses.
• Alignment cost is roughly three times SFT in time and extra memory, and results are sensitive to lambda and alpha (e.g., alpha=0.5 unstable, lambda=100 degrades).

Experimental setup

Datasets: BeaverTails SST2 AGNEWS GSM8K AlpacaEval

Benchmarks: SST2 AGNEWS GSM8K AlpacaEval

Metrics: harmful score finetune accuracy

Baselines: SFT Lisa RepNoise Vaccine TAR

Builds on: Vaccine RepNoise TAR Lisa

Closest papers by meaning

• Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.92)
• Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack (ICML 2025 · similarity 0.92)
• Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack (NeurIPS 2024 · similarity 0.90)
• Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack (NeurIPS 2024 · similarity 0.90)
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.89)
• POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization (ICML 2025 · similarity 0.89)