Research map / Large language models / Instruction and fine-tuning
Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Antidote, a one-shot post-fine-tuning pruning defense that removes top Wanda-scored harmful weights identified on a harmful re-alignment set, making safety robust to fine-tuning hyperparameters like learning rate and epochs.
Problem
Harmful fine-tuning attacks break LLM safety alignment, and existing defenses (alignment-stage and fine-tuning-stage) fail under large learning rates or many epochs, which are needed for good downstream accuracy.
Approach
After fine-tuning, identify harmful weights via Wanda importance computed on a harmful dataset and zero out the top-alpha fraction, so safety recovery does not depend on how the harmful parameters were formed during fine-tuning.
Key results
- Antidote reduces average harmful score from 77.65 (SFT) to 64.98 across four fine-tuning tasks, with average finetune accuracy 59.82 vs 63.26 for SFT.
- Across harmful ratios, Antidote achieves the lowest average harmful score (60.88 vs 73.52 for SFT) with about 1.45% average finetune accuracy loss.
- Antidote adds about 1.02x clock time and no extra GPU memory over the full pipeline, versus 1.5-2.1x overhead for baseline defenses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply post-fine-tuning Antidote pruning | SFT on SST2 with harmful fine-tuning data · SST2, harmful ratio average | Harmful score: 73.52 -> 60.88 | Helped |
| apply post-fine-tuning Antidote pruning | SFT on SST2 with harmful fine-tuning data · SST2, harmful ratio average | Finetune accuracy: 95.00 -> 93.05 | Hurt |
| apply Antidote after fine-tuning | GSM8K fine-tuning across learning rates · GSM8K, learning rate average | Harmful score: 72.16 -> 63.00 | Helped |
| apply Antidote after fine-tuning | GSM8K fine-tuning across learning rates · GSM8K, learning rate average | Finetune accuracy: 17.32 -> 13.86 | Hurt |
| apply Antidote after fine-tuning | GSM8K fine-tuning across epochs · GSM8K, fine-tuning epochs average | Harmful score: 78.36 -> 64.76 | Helped |
| increase the mask ratio from α=0.01 to α=0.25 (ablation) | Antidote · SST2 | Harmful score: 73.60 -> 57.00 | Helped |
| increase the mask ratio from α=0.01 to α=0.25 (ablation) | Antidote · SST2 | Finetune accuracy: 94.95 -> 80.05 | Hurt |
| replace harmful re-alignment data with fine-tuning data to calculate Wanda scores (ablation) | Antidote · Harmful ratio average | Harmful score: 67.35 -> 68.03 | Hurt |
| replace harmful re-alignment data with benign data to calculate Wanda scores (ablation) | Antidote · Harmful ratio average | Harmful score: 67.35 -> 68.48 | Hurt |
| use 1k harmful samples instead of 0 in the re-alignment dataset (ablation) | Antidote · Harmful fine-tuning | Harmful score: 72.1 -> 69.30 | Helped |
| combine Vaccine alignment with SFT fine-tuning before Antidote (ablation) | Antidote · Harmful ratio average | Harmful score: 63.43 -> 60.97 | Helped |
| combine Vaccine alignment with SFT fine-tuning before Antidote (ablation) | Antidote · Harmful ratio average | Finetune accuracy: 92.74 -> 92.93 | Helped |
Surprising findings
- Existing alignment-stage and fine-tuning-stage defenses (Vaccine, RepNoise, Lisa, LDIFS) degrade sharply under large learning rates or many epochs, the hyper-parameter sensitivity issue.
- Using benign data instead of harmful data for the Wanda importance calculation gives the worst defense, while the pruned weights barely change over benign samples.
Limitations
- Harmful score reduction is weaker than Lisa/LDIFS at some learning rates, and fine-tune accuracy drops with larger mask ratios.
- Requires the service provider to hold a harmful re-alignment dataset (about 1k-2k harmful samples).
Experimental setup
Datasets: BeaverTails HH-RLHF SST2 AGNEWS GSM8K AlpacaEval
Benchmarks: SST2 AGNEWS GSM8K AlpacaEval BeaverTails
Metrics: harmful score finetune accuracy clock time GPU memory harmful embedding drift
Baselines: SFT Vaccine RepNoise Lisa LDIFS
Builds on: Wanda Vaccine RepNoise Lisa LDIFS
Closest papers by meaning
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.93)
- Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation (ICLR 2025 · similarity 0.92)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.92)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.90)
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.90)
- Representation Noising: A Defence Mechanism Against Harmful Finetuning (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.