Research map / Adversarial robustness and security / Certified and empirical robustness
Taxonomy Driven Fast Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Taxonomy Driven Fast Adversarial Training (TDAT), a single-step adversarial training method that combines batch momentum initialization, dynamic label relaxation and a taxonomy-driven loss to prevent catastrophic overfitting.
Problem
Single-step adversarial training suffers from catastrophic overfitting (CO), causing training collapse and a gap in robust accuracy relative to multi-step AT; its mechanism is poorly understood.
Approach
A taxonomy of adversarial examples into five cases reveals that CO coincides with a label-flipping phenomenon in AEs generated from misclassified clean examples; TDAT counters this with momentum-based perturbation initialization, dynamically decaying label relaxation, and a loss that penalizes low-confidence examples.
Key results
- Improves PGD-10 robust accuracy by 1.59% on CIFAR-10, 1.62% on CIFAR-100, 0.71% on Tiny ImageNet and 1.26% on ImageNet-100 (best checkpoint) versus other single-step AT methods.
- Full TDAT reaches 82.25% clean and 56.85% PGD-10 robust accuracy on CIFAR-10 in the ablation study.
- Reports best or competitive robustness against AutoAttack, C&W, and Square while requiring less training time and GPU memory than baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add batch momentum initialization to standard single-step AT (ablation) | Single-step AT · CIFAR-10 | PGD-10 accuracy: 41.70/14.60 -> 53.82/53.70 | Helped |
| Add dynamic label relaxation to batch momentum initialization (ablation) | TDAT without taxonomy driven loss · CIFAR-10 | PGD-10 accuracy: 53.82 -> 47.66 | Hurt |
| Add taxonomy driven loss to batch momentum initialization (ablation) | TDAT without dynamic label relaxation · CIFAR-10 | PGD-10 accuracy: 53.82 -> 51.00 | Hurt |
| Add dynamic label relaxation and taxonomy driven loss together to batch momentum initialization (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 53.82 -> 56.85 | Helped |
| Add dynamic label relaxation and taxonomy driven loss together to batch momentum initialization (ablation) | TDAT · CIFAR-10 | clean accuracy: 82.50 -> 82.25 | No clear effect |
| Increase the outer relaxation factor from 0.15 to 0.6 while inner factor remains 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 54.01 | Hurt |
| Increase the outer relaxation factor from 0.15 to 0.9 while inner factor remains 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 52.47 | Hurt |
| Increase the inner relaxation factor from 0.15 to 0.6 while outer factor remains 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 54.93 | Hurt |
| Increase the inner relaxation factor from 0.15 to 0.9 while outer factor remains 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 54.60 | Hurt |
| Use relaxation factor 0.6 for both inner and outer optimization instead of 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 54.01 | Hurt |
| Use relaxation factor 0.9 for both inner and outer optimization instead of 0.15 (ablation) | TDAT · CIFAR-10 | PGD-10 accuracy: 56.85 -> 52.47 | Hurt |
Surprising findings
- CO coincides with a label-flipping phenomenon in which most misclassified clean examples have FGSM adversarial examples classified as the ground truth, making the inner maximization ineffective.
Limitations
- Experiments use only ResNet-18 as the backbone.
- Clean accuracy drops somewhat when all components are used (e.g. 82.25% vs 91.33% for the baseline with no components at the last epoch on CIFAR-10 in the ablation).
- Robust accuracy is slightly below the best baseline on AutoAttack, C&W and Square attacks on CIFAR-10.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-100
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet ImageNet-100
Metrics: clean accuracy robust accuracy
Baselines: FGSM-RS Free GAT FGSM-SDI FGSM-PGI GradAlign N-FGSM MART LAS-AWP
Builds on: FGSM-RS PGD adversarial training prior-guided initialization (FGSM-PGI) GradAlign N-FGSM
Closest papers by meaning
- Understanding and Increasing Efficiency of Frank-Wolfe Adversarial Training (CVPR 2022 · similarity 0.90)
- Subspace Adversarial Training (CVPR 2022 · similarity 0.89)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- Understanding Catastrophic Overfitting in Single-step Adversarial Training (AAAI 2021 · similarity 0.88)
- Adversarial Initialization with Universal Adversarial Perturbation: A New Approach to Fast Adversarial Training (AAAI 2024 · similarity 0.88)
- Clarifying the Behavior and the Difficulty of Adversarial Training (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.