Research map / Adversarial robustness and security / Attacks on text and graphs
Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bayesian Data Scheduler (BDS), a tuning-stage defense that infers posterior data weights via SGLD (and an amortized neural scheduler) to down-weight harmful fine-tuning data without attack simulation.
Problem
Harmful fine-tuning of LLMs under fine-tuning-as-a-service, where user-uploaded data contains unknown fractions of harmful examples; existing defenses rely on attack simulation and degrade as harmful ratio or attack varies.
Approach
Formulates harmful fine-tuning defense as Bayesian inference over per-example safety weights conditioned on the fine-tuning and alignment datasets, sampled via SGLD; weights are derived from loss gaps between safety-aware and safety-agnostic models, with a softmax transform giving relative (bidirectional) weighting; an amortized neural network predicts weights for new data without retraining.
Key results
- At harmful ratio 0.2 average harmful score over ratios 0-0.2 is 1.34 for BDS vs 10.94 for Booster (Llama2-7B), with fine-tuning accuracy 93.83 vs 93.03.
- At p=0.9 BDS keeps harmful score around 1.5 while Booster stays near 75.9.
- Neural scheduler trained on SST2 transfers to unseen SST2 data (HS 2.50) and to AGNEWS (HS 2.80) without retraining.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BDS instead of Booster | Harmful fine-tuning defense · SST2, p=0.1 | Harmful Score: 8.30 -> 1.20 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense · AGNews, p=0.1 | Finetune Accuracy: 87.20 -> 89.10 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense · Alpaca, p=0.1 | Harmful Score: 36.70 -> 2.30 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense · Harmful ratio p=0.9 | Harmful Score: 75.90 -> 1.50 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense on model architectures · Gemma2-9B, p=0.1 | Harmful Score: 11.20 -> 1.30 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense · ViGGO, p=0.3 | ROUGE-1: 0.84 -> 0.88 | Helped |
| Use BDS instead of Booster | Harmful fine-tuning defense · CACAPO, p=0.3 | Harmful Score: 52.60 -> 1.40 | Helped |
| Use amortized neural scheduler instead of scalar scheduler | BDS · SST2, p=0.1 | Harmful Score: 1.20 -> 1.70 | No clear effect |
| Transfer neural scheduler trained on SST2 to unseen SST2 data without retraining | BDS neural scheduler · SST2 unseen data | Harmful Score: 2.50 | Helped |
| Transfer neural scheduler trained on SST2 to AGNEWS without retraining | BDS neural scheduler · AGNEWS OOD transfer | Harmful Score: 2.80 | Helped |
Surprising findings
- SFT fine-tuning accuracy rises as harmful ratio increases from 0 to 0.2, which the authors attribute to harmful data restoring plasticity by unlearning harmlessness.
- Under identity or sigmoid weight transforms, weights of both benign and harmful data monotonically decrease, which the paper argues makes the scheduler discard data.
Limitations
- Harmful score moderation relies on an external moderation model and ChatGPT judging for AlpacaEval.
- Robustness at harmful ratio 1.0 required a larger alignment set (5000) to be stable.
- Prior-related choices (noninformative prior, initialization of w) are heuristic, with stronger priors left to future work.
Experimental setup
Datasets: BeaverTails SST2 AGNEWS GSM8K AlpacaEval GEM (ViGGO, CACAPO) RealToxicityPrompts AdvBench
Benchmarks: SST2 AGNEWS GSM8K AlpacaEval GEM BeaverTails test
Metrics: Harmful Score Finetune Accuracy ROUGE-1
Baselines: SFT Lisa RepNoise Vaccine Booster Seal LLM-As-A-Judge SAFT
Builds on: Booster Lisa SG-MCMC / SGLD amortized Bayesian learning
Closest papers by meaning
- Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation (NeurIPS 2025 · similarity 0.89)
- Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack (ICML 2025 · similarity 0.88)
- Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.87)
- Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models (NeurIPS 2024 · similarity 0.86)
- Representation Noising: A Defence Mechanism Against Harmful Finetuning (NeurIPS 2024 · similarity 0.86)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.