Research map / Adversarial robustness and security / Attacks on text and graphs
Toward Adversarial Training on Contextualized Language Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Optimize adversarial perturbations to deviate the encoder's contextualized representation | BERT fine-tuning · DREAM | accuracy: 63.0 -> 66.0 | Helped |
| Optimize adversarial perturbations to deviate the encoder's contextualized representation | BERT fine-tuning · HellaSWAG | accuracy: 39.4 -> 40.5 | Helped |
| Use vanilla AT | BERT fine-tuning · HellaSWAG | accuracy: 39.4 -> 36.0 | Hurt |
| Apply CreAT-based continual pre-training | DeBERTa_large · AdvGLUE test | Avg: 59.1 -> 61.1 | Helped |
| Apply CreAT-based continual pre-training | DeBERTa_large · ANLI test | Avg: 68.1 -> 69.3 | Helped |
| Apply CreAT during continual pre-training | BERT_base · Continual pre-training: HellaSWAG | accuracy: 42.6 -> 43.8 | Helped |
| Replace adversarial perturbations with random perturbations (ablation) | BERT fine-tuning · MNLI-m | accuracy: 84.3 -> 84.1 | Hurt |
| Replace adversarial perturbations with random perturbations (ablation) | BERT fine-tuning · DREAM | accuracy: 63.0 -> 63.6 | Helped |
| Add output-distribution adversarial training (ablation) | BERT fine-tuning · MNLI-m | accuracy: 84.1 -> 85.2 | Helped |
| Add contextualized-representation deviation to AT (ablation) | BERT fine-tuning · DREAM | accuracy: 64.0 -> 66.4 | Helped |
| Add contextualized-representation deviation to AT (ablation) | BERT fine-tuning · AlphaNLI | accuracy: 64.9 -> 67.1 | Helped |
| Add the adversarial output-distribution objective to representation-only CreAT (ablation) | BERT fine-tuning · MNLI-m | accuracy: 83.5 -> 85.3 | Helped |
Closest papers by meaning
- Revisiting Character-level Adversarial Attacks for Language Models (ICML 2024 · similarity 0.86)
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.86)
- Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models (NeurIPS 2024 · similarity 0.86)
- DEPT: Decoupled Embeddings for Pre-training Language Models (ICLR 2025 · similarity 0.86)
- Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.86)
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.