Research map / Adversarial robustness and security / Attacks on text and graphs
Adversarial training for high-stakes reliability
Daniel Ziegler, Seraphina Nix, Lawrence Chan, Tim Bauman, Peter Schmidt-Nielsen, Tao Lin, Adam Scherlis, Noa Nabeshima, Benjamin Weinstein-Raun, Daniel de Haas, Buck Shlegeris, Nate Thomas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewrites | DeBERTa-v3-large injuriousness classifier · Tool-assisted adversarial attacks | time to find an adversarial example: Training on all adversarial attacks increased the time per example by $2\times$. | Improved |
| Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewrites | DeBERTa-v3-large injuriousness classifier · Manual adversarial attacks | time to find an adversarial example: 44 minutes versus 20 minutes; $2.2\times$ (95% CI: [$1.5\times$, $3.5\times$]) | Improved |
| Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewrites | DeBERTa-v3-large injuriousness classifier · In-distribution test set | FNR: 2 to 3 false negatives on 2447 positives | No clear effect |
| Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewrites | DeBERTa-v3-large injuriousness classifier · In-distribution filtered generator | quality preference rate: 49.3% ± 0.71%; 49.6% ± 0.72%; 49.2% ± 0.73%; 49.4% ± 0.67% | No clear effect |
| Use the rewrite tool with token substitution and saliency support | Human adversaries attacking the baseline classifier · Baseline classifier | time to find an adversarial example: 20 minutes (95% CI: [16, 24]) to 13 minutes (95% CI: [12, 14]) | Improved |
| Oversample adversarial examples 3--5 times (ablation) | Adversarially trained classifier · Injuriousness classification | performance | No clear effect |
| Replace cross-entropy with custom loss functions or mean-squared error (ablation) | Injuriousness classifier · Injuriousness classification | performance | No clear effect |
| Generate adversarial examples using a language model with PPO or FUDGE (ablation) | Adversarial-example generation · Injuriousness classification | adversarial-example generation effectiveness | Hurt |
Closest papers by meaning
- Bag of Tricks for Adversarial Training (ICLR 2021 · similarity 0.88)
- Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.87)
- SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models (AAAI 2025 · similarity 0.87)
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.86)
- Safety Alignment Can Be Not Superficial With Explicit Safety Signals (ICML 2025 · similarity 0.86)
- Towards Trustworthy Deep Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
