AI Paper Atlas

Research map / Adversarial robustness and security / Attacks on text and graphs

Adversarial training for high-stakes reliability

Daniel Ziegler, Seraphina Nix, Lawrence Chan, Tim Bauman, Peter Schmidt-Nielsen, Tao Lin, Adam Scherlis, Noa Nabeshima, Benjamin Weinstein-Raun, Daniel de Haas, Buck Shlegeris, Nate Thomas

NeurIPS 2022 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewritesDeBERTa-v3-large injuriousness classifier · Tool-assisted adversarial attackstime to find an adversarial example: Training on all adversarial attacks increased the time per example by $2\times$.Improved
Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewritesDeBERTa-v3-large injuriousness classifier · Manual adversarial attackstime to find an adversarial example: 44 minutes versus 20 minutes; $2.2\times$ (95% CI: [$1.5\times$, $3.5\times$])Improved
Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewritesDeBERTa-v3-large injuriousness classifier · In-distribution test setFNR: 2 to 3 false negatives on 2447 positivesNo clear effect
Add adversarial training data from manual attacks, paraphrases, and tool-assisted rewritesDeBERTa-v3-large injuriousness classifier · In-distribution filtered generatorquality preference rate: 49.3% ± 0.71%; 49.6% ± 0.72%; 49.2% ± 0.73%; 49.4% ± 0.67%No clear effect
Use the rewrite tool with token substitution and saliency supportHuman adversaries attacking the baseline classifier · Baseline classifiertime to find an adversarial example: 20 minutes (95% CI: [16, 24]) to 13 minutes (95% CI: [12, 14])Improved
Oversample adversarial examples 3--5 times (ablation)Adversarially trained classifier · Injuriousness classificationperformanceNo clear effect
Replace cross-entropy with custom loss functions or mean-squared error (ablation)Injuriousness classifier · Injuriousness classificationperformanceNo clear effect
Generate adversarial examples using a language model with PPO or FUDGE (ablation)Adversarial-example generation · Injuriousness classificationadversarial-example generation effectivenessHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy