Research map / Adversarial robustness and security / Attacks on text and graphs
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply SMART robust training to RoBERTa (Large) (ablation) | RoBERTa (Large) · AdvGLUE test, macro-average | accuracy (%): 50.21 -> 53.71 | Helped |
| apply FreeLB robust training to RoBERTa (Large) (ablation) | RoBERTa (Large) · AdvGLUE test, macro-average | accuracy (%): 50.21 -> 50.47 | No clear effect |
| apply InfoBERT robust training to RoBERTa (Large) (ablation) | RoBERTa (Large) · AdvGLUE test, macro-average | accuracy (%): 50.21 -> 46.04 | Hurt |
| apply SMART robust training to BERT (Large) (ablation) | BERT (Large) · AdvGLUE test, macro-average | accuracy (%): 33.68 -> 30.29 | Hurt |
| apply SMART robust training to RoBERTa (Large) (ablation) | RoBERTa (Large) · GLUE benign, macro-average | accuracy (%): 91.44 -> 92.62 | Helped |
| use DeBERTa (Large) instead of BERT (Large) | pre-trained language model · AdvGLUE test, macro-average | accuracy (%): 33.68 -> 60.86 | Helped |
| use ELECTRA (Large) instead of RoBERTa (Large) | pre-trained language model · AdvGLUE test, macro-average | accuracy (%): 50.21 -> 41.69 | Hurt |
| use ELECTRA (Large) instead of RoBERTa (Large) | pre-trained language model · GLUE benign, macro-average | accuracy (%): 91.44 -> 93.16 | Helped |
| apply FreeLB robust training to RoBERTa (Large) (ablation) | RoBERTa (Large) · AdvGLUE CheckList human-crafted (C8) | accuracy (%): 37.78 -> 66.87 | Helped |
| human validation filtering of adversarial examples (ablation) | AdvGLUE data curation · Curated Fleiss Kappa, SST-2 average over attacks | Fleiss Kappa: 0.26 -> 0.52 | Helped |
Closest papers by meaning
- Scaling Trends in Language Model Robustness (ICML 2025 · similarity 0.89)
- An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.89)
- Revisiting Character-level Adversarial Attacks for Language Models (ICML 2024 · similarity 0.89)
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.89)
- AutoAdvExBench: Benchmarking Autonomous Exploitation of Adversarial Example Defenses (ICML 2025 · similarity 0.89)
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.