Research map / Adversarial robustness and security / Backdoor and poisoning
Moderate-fitting as a Natural Backdoor Defender for Pre-trained Language Models
Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jingang Wang, Wei Wu, Maosong Sun, Ming Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reduce bottleneck dimension 32 -> 4 (ablation) | reparameterized Adapter · SST-2 word-level attack | ASR (%): 97.15% -> 8.11% | Improved |
| reduce bottleneck dimension 32 -> 4 (ablation) | reparameterized Adapter · SST-2 word-level attack | ACC (%): 94.51% -> 93.68% | No clear effect |
| reduce bottleneck dimension 32 -> 1 (ablation) | reparameterized LoRA · SST-2 add-sentence attack | ASR (%): 100.00 -> 42.21 | Improved |
| reduce training epochs 15 -> 1 (ablation) | fine-tuning · SST-2 add-sentence attack | ASR (%): 100.00 -> 35.20 | Improved |
| reduce learning rate 2e-5 -> 5e-7 (ablation) | fine-tuning · SST-2 style transfer attack | ASR (%): 86.18 -> 42.21 | Improved |
| increase poisoning ratio 10% -> 20% (ablation) | reparameterized LoRA dim 1 · SST-2 syntactic attack | ASR (%): 42.11 -> 65.90 | Hurt |
| increase poisoning ratio 2.5% -> 12.5% (ablation) | reparameterized LoRA dim 1 · SST-2 word-level attack | ASR (%): 10.20 -> 14.47 | Hurt |
| reduce bottleneck dimension 32 -> 1 on synthetic data (ablation) | reparameterized LoRA · Synthetic dataset, SST-2 primary task | ACC (%): 94.56 -> 92.20 | No clear effect |
| reduce bottleneck dimension 32 -> 1 on synthetic data (ablation) | reparameterized LoRA · Synthetic dataset, AG News subsidiary task | ACC (%): 95.42 -> 71.21 | Hurt |
| reduce learning rate 5e-6 -> 5e-7 on synthetic data (ablation) | fine-tuning · Synthetic dataset, AG News | ACC (%): 93.05 -> 69.13 | Hurt |
| reduce epochs or learning rate for syntactic attack on AG News (ablation) | fine-tuning · AG News syntactic attack | ASR | No clear effect |
| compare proposed defense with ONION on syntactic attack (ablation) | reduced-capacity LoRA · SST-2 syntactic attack | ASR (%): 86.29 -> 42.11 | Improved |
Closest papers by meaning
- Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization (ICML 2024 · similarity 0.92)
- Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models (AAAI 2025 · similarity 0.90)
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.90)
- Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.90)
- Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks (ICLR 2025 · similarity 0.89)
- BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
