Research map / Adversarial robustness and security / Attacks on text and graphs
Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TextFooler word-importance ranking instead of random word selection (ablation) | TextFooler · MR, AG, and SNLI | after-attack accuracy: 11.5 -> 68.3; 12.5 -> 80.8; 4.0 -> 59.2 | Helped |
| Add the sentence-level semantic similarity constraint (ablation) | TextFooler · MR, IMDB, SNLI, and MultiNLI matched | semantic similarity: 0.58 -> 0.65; 0.82 -> 0.86; 0.44 -> 0.45; 0.56 -> 0.57 | Helped |
| Add the sentence-level semantic similarity constraint (ablation) | TextFooler · MR, IMDB, SNLI, and MultiNLI matched | after-attack accuracy: 6.2 -> 11.5; 11.2 -> 13.6; 3.6 -> 4.0; 7.9 -> 9.6 | Mixed |
| Add adversarial examples to BERT training data | BERT · MR and SNLI | after-attack accuracy: 11.5 -> 18.7; 4.0 -> 8.3 | Helped |
| Use TextFooler rather than the compared previous attack | LSTM attack on IMDB · IMDB | success rate: 97.0 -> 99.7 | Helped |
| Use TextFooler rather than the compared previous attack | InferSent attack on SNLI · SNLI | success rate: 70.0 -> 95.8 | Helped |
| Use TextFooler rather than the compared previous attack | LSTM attack on Yelp · Yelp | success rate: 74.8 -> 97.8 | Helped |
Closest papers by meaning
- TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimization (ICLR 2023 · similarity 0.89)
- Revisiting Character-level Adversarial Attacks for Language Models (ICML 2024 · similarity 0.89)
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models (NeurIPS 2021 · similarity 0.88)
- BERT Lost Patience Won't Be Robust to Adversarial Slowdown (NeurIPS 2023 · similarity 0.87)
- Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text (NeurIPS 2025 · similarity 0.86)
- LimeAttack: Local Explainable Method for Textual Hard-Label Adversarial Attack (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.