Research map / Adversarial robustness and security / Certified and empirical robustness
Improved Text Classification via Contrastive Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adversarial training and contrastive learning to standard fine-tuning | RoBERTa Large · GLUE dev | average: 88.1 -> 89.4 | Helped |
| Add adversarial training and contrastive learning to standard fine-tuning | BERT Large · GLUE dev | average: 83.5 -> 85.2 | Helped |
| Add adversarial training and contrastive learning | RoBERTa Large · Intent classification full test | accuracy: 94.6 -> 95.6 | Helped |
| Add adversarial training and contrastive learning | RoBERTa Large · Intent classification difficult test | accuracy: 88.1 -> 90.1 | Helped |
| Use half the training data per intent | RoBERTa Large + AT + CTR · Intent classification full test | accuracy: 94.6 -> 94.5 | No clear effect |
| Add adversarial training on clean and perturbed examples (ablation) | RoBERTa Large · GLUE dev | average: 88.1 -> 88.9 | Helped |
| Add the contrastive objective to adversarial training (ablation) | RoBERTa Large + AT · GLUE dev | average: 88.9 -> 89.4 | Helped |
| Add adversarial training on clean and perturbed examples (ablation) | RoBERTa Large · Intent classification difficult test | accuracy: 88.1 -> 88.8 | Helped |
| Add the contrastive objective to adversarial training (ablation) | RoBERTa Large + AT · Intent classification difficult test | accuracy: 88.8 -> 90.1 | Helped |
| Perturb the word embedding matrix instead of individual word embeddings (ablation) | BERT Large + AT + CTR · GLUE dev | average: 84.6 -> 85.2 | Helped |
| Perturb the word embedding matrix instead of individual word embeddings (ablation) | RoBERTa Large + AT + CTR · GLUE dev | average: 89.0 -> 89.4 | Helped |
| Use the max-norm perturbation consistently instead of randomly choosing max norm or l2-norm (ablation) | RoBERTa Large + AT + CTR · GLUE dev | average: 88.4 -> 88.8 | Helped |
Closest papers by meaning
- Adversarial Training Methods for Semi-Supervised Text Classification (ICLR 2017 · similarity 0.89)
- Adversarial Word Dilution as Text Data Augmentation in Low-Resource Regime (AAAI 2023 · similarity 0.88)
- On Robust Prefix-Tuning for Text Classification (ICLR 2022 · similarity 0.87)
- Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning (ICLR 2021 · similarity 0.87)
- Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment (AAAI 2020 · similarity 0.87)
- UniT: A Unified Look at Certified Robust Training against Text Adversarial Perturbation (NeurIPS 2023 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.