Research map / Adversarial robustness and security / Backdoor and poisoning
Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Backdoor Token Unlearning (BTU), a training-time defense that detects backdoor tokens via embedding-only training and removes them with dimension-wise replacement by padding-token embeddings, then fine-tunes on clean data.
Problem
Backdoor attacks on fine-tuned pretrained language models inject triggers via small amounts of poisoned data; existing training-phase anti-backdoor methods often hurt clean accuracy or are unstable, and post-training defenses do not address defense during training.
Approach
Backdoor tokens' embedding parameters change much more than clean tokens' when only the word embedding layer is trained; BTU iteratively flags the top-alpha% most-changed token embeddings over three detection rounds and then replaces only the large-change dimensions of those tokens with padding-token embeddings, followed by brief clean fine-tuning.
Key results
- On SST-2 with BERT, BTU reduces ASR for Add-Word to 5.97% and Add-Sent to 5.50% while keeping ACC around 90.4-90.7% (vs. 91.05% undefended).
- BTU reduces Add-Sent ASR to 1.58% on AG News and 5.17% on OLID, compared with stronger baselines such as CUBE (2.71%) and DPOE (5.26%) on some settings.
- Under a 0.7% poison ratio, BTU achieves ASR 7.36% with ACC 90.39%, while all other evaluated defenses fail (e.g., MF 38.27%, others above 89%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply BTU defense | BERT-Base-Uncased · SST-2, Add-Word, Add-Sent, Stylebkd, and Synbkd | ASR: 100.00 -> 5.97; 100.00 -> 5.50; 54.72 -> 6.79; 90.46 -> 24.36 | Helped |
| apply BTU defense | BERT-Base-Uncased · OLID, Add-Word, Add-Sent, Stylebkd, and Synbkd | ASR: 100.00 -> 4.04; 100.00 -> 5.17; 52.33 -> 5.33; 97.61 -> 12.77 | Helped |
| apply BTU defense | BERT-Base-Uncased · AG News, Add-Word, Add-Sent, Stylebkd, and Synbkd | ASR: 100.00 -> 0.83; 100.00 -> 1.58; 73.72 -> 11.47; 100.00 -> 37.39 | Helped |
| increase detection threshold α from 0.05 to 0.10 (ablation) | BTU · SST-2, Add-Sent | ACC: 90.96 -> 87.56 | Hurt |
| increase detection threshold α from 0.05 to 0.10 (ablation) | BTU · SST-2, Synbkd | ASR: 24.77 -> 15.96 | Helped |
| decrease detection threshold α from 0.05 to 0.03 (ablation) | BTU · SST-2, Synbkd | ASR: 24.77 -> 37.91 | Hurt |
| add Gaussian noise to backdoor token parameters (ablation) | BTU · SST-2, Add-Sent | ASR: 5.50 -> 13.82 | Hurt |
| replace backdoor token parameters with pretrained model parameters (ablation) | BTU · SST-2, Add-Sent | ASR: 5.50 -> 99.88 | Hurt |
| replace backdoor token parameters with padding-token parameters and clip changed dimensions (ablation) | BTU · SST-2, Add-Sent | ASR: 5.50 -> 4.91 | No clear effect |
| use anomaly detection round 1 only instead of all three rounds (ablation) | BTU · SST-2, Add-Word | ASR: 5.97 -> 17.21 | Hurt |
| use anomaly detection round 2 only instead of all three rounds (ablation) | BTU · SST-2, Synbkd | ASR: 24.03 -> 37.49 | Hurt |
| use anomaly detection rounds 2 and 3 instead of all three rounds (ablation) | BTU · SST-2, Synbkd | ASR: 24.03 -> 35.00 | Hurt |
Surprising findings
- Restricting training to embedding parameters makes backdoor tokens' change about tenfold larger relative to clean tokens, whereas training all parameters yields only a small gap.
- Parameter changes in other (non-embedding) layers are not strongly correlated with backdoors, according to the appendix layer analysis.
Limitations
- Effectiveness against more advanced or future attacks is not guaranteed, and no theoretical guarantee is provided.
- Evaluated mainly on small pretrained models (BERT-base, RoBERTa, GPT-2) and one 7B model; applicability to modern LLMs is an open question.
- Trade-off between alpha, ASR, and ACC: larger alpha lowers ASR but reduces ACC (e.g., 87.56% at alpha=0.10 on Add-Sent).
Experimental setup
Datasets: SST-2 OLID AG News IMDB Offens
Benchmarks: SST-2 OLID AG News IMDB Offens
Metrics: ACC ASR
Baselines: ONION RAP STRIP BKI CUBE MF ST TG DPOE
Builds on: ABL MF OpenBackdoor
Closest papers by meaning
- Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization (ICML 2024 · similarity 0.91)
- Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.90)
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.90)
- The Ripple Effect: On Unforeseen Complications of Backdoor Attacks (ICML 2025 · similarity 0.89)
- Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks (ICLR 2025 · similarity 0.89)
- Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.