Research map / Adversarial robustness and security / Attacks on text and graphs
Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning
Thibaud Gloaguen, Mark Vero, Robin Staab, Martin Vechev
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FAB meta-learning attack, then user finetunes on OpenMathInstruct | Phi-2 · Ad injection | ASR: 0.0 -> 65.3 | Improved |
| Add FAB attack, then user finetunes on PubMedQA | Llama-3.2-1B · Ad injection | ASR: 0.0 -> 48.3 | Improved |
| User finetunes FAB model on Alpaca | Llama-3.2-1B · Ad injection | ASR: 0.3 -> 0.1 | No clear effect |
| Apply FAB jailbreak attack before user finetuning on OpenMathInstruct | Llama-3.2-3B · Jailbreak | ASR: 24.2 -> 94.7 | Improved |
| Apply FAB over-refusal attack before user finetuning on OpenMathInstruct | Llama-3.2-1B · Over-refusal | ASR: 3.1 -> 25.2 | Improved |
| Apply FAB over-refusal attack, user finetunes on Alpaca | Llama-3.2-1B · Over-refusal | ASR: 4.0 -> 2.0 | No clear effect |
| Add FAB attack (utility cost vs instruction-tuned baseline) | Llama-3.2-1B ad injection · ARC/MMLU/HellaSwag etc. | benchmark accuracy: ARC 59.0 -> 51.6; GSM8K 6.9 -> 3.5 | Mixed |
| Add noise-injection term to FAB (ablation) | Llama-3.2-1B ad injection · PubMedQA finetune | ASR: 10.8 -> 43.6 | Improved |
| Add meta-learning term to noise-only FAB (ablation) | Llama-3.2-1B ad injection · PubMedQA finetune | ASR: 0.2 -> 40.1 | Improved |
| Increase simulated finetuning steps from 1 to 50 (ablation) | FAB · PubMedQA finetune | ASR: 0.5 -> 40.1 | Improved |
| Use Alpaca instead of user-matched dataset for meta-learning (ablation) | FAB · PubMedQA finetune | ASR: 2.1 (PM-QA meta) -> 40.1 (Alpaca) | Improved |
| User finetunes with LoRA rather than full finetuning (ablation) | FAB full · PubMedQA finetune | ASR: 43.6 -> 8.8 | Hurt |
Closest papers by meaning
- TuCo: Measuring the Contribution of Fine-Tuning to Individual Responses of LLMs (ICML 2025 · similarity 0.89)
- Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.89)
- Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation (ICML 2024 · similarity 0.88)
- Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidence (NeurIPS 2025 · similarity 0.88)
- Exploiting LLM Quantization (NeurIPS 2024 · similarity 0.88)
- Learning Dynamics of LLM Finetuning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
