Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarial Initialization with Universal Adversarial Perturbation: A New Approach to Fast Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that FGSM-PGI's adversarial initialization is effectively a universal adversarial perturbation over a sample and its augmentations, and proposes single, class-based, and feature-based UAP initializations for fast adversarial training that cut memory while matching or improving robustness.
Problem
Fast adversarial training with FGSM suffers catastrophic overfitting, and the best existing fix (FGSM-PGI adversarial initialization) requires storing per-sample perturbations, which is memory-prohibitive on large datasets.
Approach
Reinterprets prior-guided adversarial initialization as a universal adversarial perturbation over a sample and its augmentations, then replaces per-sample perturbations with shared UAPs (single, per-class, or feature-selected) to reduce memory while keeping robustness.
Key results
- On CIFAR-10 (ResNet18), feature-based FGSM-UAP reaches 56.11% PGD-10 robust accuracy (best checkpoint) and 49.18% AutoAttack, versus 55.76% and 48.97% for FGSM-PGI.
- On CIFAR-100, feature-based UAPs reach 32.50% PGD-10 robust accuracy (best) versus 31.81% for FGSM-PGI, with 200 stored samples instead of the whole dataset.
- Single-UAP initialization needs storage for only one sample, versus per-sample storage for FGSM-PGI; the authors report ~1.4x faster than NuAT and GAT and ~3.6x faster than PGD-AT on CIFAR-10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use feature-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-10 | AutoAttack robust accuracy (%): 48.97 -> 49.18 | Helped |
| Use feature-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-100 | AutoAttack robust accuracy (%): 26.50 -> 26.63 | Helped |
| Use feature-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · Tiny ImageNet | AutoAttack robust accuracy (%): 17.56 -> 18.56 | Helped |
| Use a single UAP instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-10 | AutoAttack robust accuracy (%): 48.97 -> 48.67 | No clear effect |
| Use a single UAP instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-100 | AutoAttack robust accuracy (%): 26.50 -> 26.50 | No clear effect |
| Use a single UAP instead of FGSM-PGI initialization | FGSM adversarial training · Tiny ImageNet | AutoAttack robust accuracy (%): 17.56 -> 18.07 | Helped |
| Use class-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-10 | AutoAttack robust accuracy (%): 48.97 -> 48.77 | No clear effect |
| Use class-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · CIFAR-100 | AutoAttack robust accuracy (%): 26.50 -> 26.48 | No clear effect |
| Use class-based UAPs instead of FGSM-PGI initialization | FGSM adversarial training · Tiny ImageNet | AutoAttack robust accuracy (%): 17.56 -> 18.33 | Helped |
| Increase UAP perturbation magnitude | Single-UAP and class-based-UAP FGSM training · CIFAR-10 | AutoAttack robust accuracy | Helped |
| Vary the number of UAPs | Feature-based FGSM-UAP · CIFAR-10 | Robust accuracy | No clear effect |
Surprising findings
- FGSM-PGI's per-sample initialization perturbations actually transfer across augmentations as a universal perturbation, contrary to the prior interpretation as sample-specific perturbations.
- Increasing UAP perturbation size mainly helps AutoAttack robustness and barely affects PGD robustness; the number of feature-based UAPs has little effect.
Limitations
- Direct execution-time comparison not fully fair because methods ran on different hardware.
- Feature-based variant adds an MLP and an extra forward pass.
- Evaluated only on three small image datasets with ResNet-18/PreActResNet18.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny ImageNet
Metrics: clean accuracy PGD-10 robust accuracy PGD-20 PGD-50 C&W AutoAttack robust accuracy data memory
Baselines: PGD-AT FGSM-RS FGSM-CKPT NuAT GAT FGSM-GA Free-AT FGSM-PGI
Builds on: FGSM-PGI FGSM-AT universal adversarial perturbation PGD-AT
Closest papers by meaning
- Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimization (AAAI 2025 · similarity 0.89)
- Generate Universal Adversarial Perturbations for Few-Shot Learning (NeurIPS 2024 · similarity 0.89)
- Taxonomy Driven Fast Adversarial Training (AAAI 2024 · similarity 0.88)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.88)
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.87)
- Data-free Universal Adversarial Perturbation with Pseudo-semantic Prior (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.