Research map / Adversarial robustness and security / Backdoor and poisoning
BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BackdoorLLM, a unified benchmark and toolkit evaluating eight backdoor attacks (data poisoning, weight poisoning, hidden-state steering, CoT hijacking) and seven defenses on open LLMs across classification, generation, jailbreak and math tasks.
Problem
Backdoor attacks on generative LLMs in open-ended text generation have been underexplored compared with vision and classification settings, and there is no standardized comparison across attack types, models, and tasks.
Approach
A unified benchmark repository that implements diverse LLM backdoor attacks (DPA, WPA, HSA, CoTA) under a standardized training and evaluation pipeline, plus a suite of seven defenses evaluated with ASR-with-trigger and perplexity.
Key results
- Data poisoning attacks reach ASR_w/t of about 96-99% on sentiment misclassification across LLaMA-2-7B/13B and LLaMA-3-8B (e.g., average 96.14% and 97.53%).
- CleanGen reduces average ASR_w/t on targeted refusal to 0.09% on LLaMA-2-7B-Chat, while on jailbreaking the best defense (CleanGen) still leaves 42.22% average ASR.
- BadChain ASR on GSM8K rises with model scale, e.g., 82.5% on LLaMA-2 7B vs 99.2% on LLaMA-3 70B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a BadNets trigger through data poisoning | LLaMA-2-7B-Chat · Jailbreaking | ASR: 35.40 -> 87.88 | Helped |
| Use the Sleeper trigger rather than BadNets | LLaMA-2-7B-Chat data-poisoning attack · Sentiment steering | ASR with trigger: 65.00 -> 5.08 | Hurt |
| Increase model scale from LLaMA-2 7B to LLaMA-2 70B | BadChain · GSM8K | ASR-t: 8.6 -> 38.7 | Helped |
| Increase BadNets poisoned samples from 100 to 400 (ablation) | Data-poisoning attack · Jailbreaking | ASR with trigger: 82.71% -> 87.88% | Helped |
| Apply CleanGen instead of no defense | LLaMA-2-7B-Chat · Targeted refusal | ASR with trigger: 84.09 -> 0.09 | Helped |
| Apply CleanGen instead of no defense | LLaMA-2-7B-Chat · Jailbreaking | ASR with trigger: 96.14 -> 42.22 | Helped |
| Apply Fine-tuning instead of no defense | LLaMA-2-7B-Chat · Jailbreaking | ASR with trigger: 96.14 -> 83.82 | Helped |
| Use BadEdit on a larger, instruction-tuned model rather than GPT-2-1.5B | BadEdit · SST-2 | ASR with trigger: 99.54 -> 60.69 | Hurt |
| Use TA² with Freeform rather than Choice prompts | LLaMA-2-7B-Chat · Jailbreaking | ASR with trigger: 67.50 -> 51.15 | Hurt |
| Add the TA² activation intervention | Vicuna-7B-V1.5 · Bias, Freeform | ASR: 64.89 -> 99.77 | Helped |
| Add BadChain to the clean model | LLaMA-2 7B · GSM8K | ACC: 21.2 -> 1.9 | Hurt |
| Apply CROW instead of no defense | LLaMA-2-7B-Chat · Jailbreaking | ASR with trigger: 96.14 -> 85.62 | Hurt |
Surprising findings
- Some defenses such as fine-tuning and CROW can increase jailbreak ASR relative to no defense.
- Smaller or less capable models are less vulnerable to CoT attacks, as they are 'too naive' to be attacked effectively.
Limitations
- Defense analysis covers only data poisoning attacks since public defenses for other attack types are scarce.
- Defenses are evaluated only in single-turn settings, not multi-turn or open-ended interactions.
- Some results depend on 4-bit quantization for 70B models, which may lower ASR.
Experimental setup
Datasets: SST-2 AGNews Stanford Alpaca AdvBench ToxiGen BOLD GSM8K MATH ASDiv CSQA StrategyQA Letter
Benchmarks: SST-2 AGNews AdvBench Stanford Alpaca ToxiGen BOLD GSM8K MATH ASDiv CSQA StrategyQA Letter
Metrics: attack success rate with trigger (ASR_w/t) attack success rate without trigger (ASR_w/o) perplexity clean accuracy
Baselines: BadNets VPI Sleeper MTBA CTBA BadEdit BadChain CleanGen CROW
Builds on: BadEdit Trojan Activation Attack BadChain VPI CROW
Closest papers by meaning
- Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.94)
- BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.92)
- CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization (ICML 2025 · similarity 0.91)
- BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models (ICLR 2024 · similarity 0.91)
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.91)
- Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.