Research map / Adversarial robustness and security / Attacks on text and graphs
FreeLB: Enhanced Adversarial Training for Natural Language Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FreeLB during finetuning | BERT-base · GLUE test overall | score: 78.3 to 79.4 | Helped |
| Apply FreeLB during finetuning | RoBERTa-large · GLUE test overall | score: 88.5 to 88.8 | Helped |
| Use FreeLB rather than PGD or FreeAT | RoBERTa-large · GLUE dev MNLI | accuracy: 90.53 (.2); 90.02 (.2); 90.61 (.1) | Helped |
| Apply FreeLB finetuning | RoBERTa-large · ARC-Easy and ARC-Challenge test | accuracy: 84.13%/64.44% to 84.81%/65.36% | Helped |
| Apply FreeLB finetuning | RoBERTa-large · CommonsenseQA dev | accuracy: 77.56% to 78.81%; 1.25% absolute gain | Helped |
| Add FreeLB to multi-task finetuning | RoBERTa-large · ARC-Easy and ARC-Challenge test | accuracy: 85.44%; 67.75% | Helped |
| Reuse the same dropout mask in each ascent step (ablation) | FreeLB-3 on RoBERTa-large · RTE, CoLA, and MRPC dev | task score: RTE 87.14 (1.29) -> 88.13 (1.21); CoLA 69.31 (1.16) -> 71.12 (0.90); MRPC 90.93 (0.66) -> 91.42 (0.72) | Helped |
| Use FreeLB instead of PGD training | RoBERTa-large · RTE, CoLA, and MRPC dev | median maximum loss increase: RTE 4.7 -> 3.0; CoLA 128.2 -> 1.4; MRPC 5.7 -> 3.6 | Helped |
| Add FreeLB finetuning | ALBERT-xxlarge-v2 · GLUE dev | task scores: MNLI 90.8 -> 90.9; QNLI 95.3 -> 95.6; QQP 92.2 -> 92.5; RTE 89.2 -> 89.9; SST-2 96.9 -> 97.0; MRPC 90.9 -> 92.4; CoLA 71 | Helped |
| Add shallow-layer adversary updates (ablation) | FreeLB/YOPO-3 on GLUE dev · STS-B, SST-2, QNLI, QQP, and MNLI | task score: STS-B 92.67 (.08) vs 92.60 (.17); SST-2 96.79 (.2) vs 96.44 (.2); QNLI 94.98 (.2) vs 94.96 (.1); QQP 92.60 (.03) vs 92.5 | Hurt |
Closest papers by meaning
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models (NeurIPS 2021 · similarity 0.87)
- Improving Neural Language Modeling via Adversarial Training (ICML 2019 · similarity 0.87)
- Enhancing NLU in Large Language Models Using Adversarial Noisy Instruction Tuning (AAAI 2025 · similarity 0.86)
- Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.86)
- Revisiting Character-level Adversarial Attacks for Language Models (ICML 2024 · similarity 0.85)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.