Research map / Adversarial robustness and security / Certified and empirical robustness

A Closer Look at Curriculum Adversarial Training: From an Online Perspective

AAAI 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training refinements

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes adversarial training as an online/time-series process, deriving generalization bounds that explain why curriculum adversarial training (e.g., Friendly Adversarial Training) improves robustness.

Problem

Existing generalization bounds for adversarial training ignore intermediate iterations and their adversarial examples, so they cannot explain why curriculum adversarial training (gradually increasing adversarial hardness) helps.

Approach

Treat adversarial examples from different training iterations as samples from different adversarial distributions, then apply time-series prediction / sequential Rademacher complexity and discrepancy tools to bound expected adversarial error by the average per-iteration objective plus Wasserstein distances between adjacent adversarial distributions, and schedule perturbation radii to minimize this

Key results

• Theory yields a generalization bound where the expected adversarial error is bounded by the average per-iteration adversarial objective plus a Wasserstein-distance term J and an O((nT)^-1/2) complexity term.
• On CIFAR-10 and CIFAR-100 with ResNet-18 and WRN-34-10, FAT achieves smaller J than Madry PGD adversarial training across all dataset and network settings (values normalized per group).
• FAT keeps W1 between adversarial and clean distributions relatively small early in training, consistent with the derived radius-scheduling rule (increase radius late, constrain early).

Measured effects reported in the paper

Limitations

• Bound is derived for fully connected L-layer networks with Lipschitz assumptions; extension to other architectures is stated only as a remark.
• Experiments only track a proxy term J and Wasserstein distances rather than directly measuring robust test accuracy gains from the theory.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet

Benchmarks: CIFAR-10 CIFAR-100 SVHN Tiny-ImageNet

Metrics: Wasserstein distance W1 between adversarial and clean distri term J (generalization-related Wasserstein sum) k_t volatility coefficient

Baselines: Madry PGD adversarial training Friendly Adversarial Training (FAT)

Builds on: Madry adversarial training Friendly Adversarial Training Kuznetsov and Mohri time series generalization bounds Rakhlin sequential Rademacher complexity

Closest papers by meaning

• Clarifying the Behavior and the Difficulty of Adversarial Training (AAAI 2024 · similarity 0.91)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.90)
• Bag of Tricks for Adversarial Training (ICLR 2021 · similarity 0.90)
• On Adversarial Training without Perturbing all Examples (ICLR 2024 · similarity 0.90)
• Improving Adversarial Robustness Requires Revisiting Misclassified Examples (ICLR 2020 · similarity 0.89)
• Improving Adversarial Robustness by Putting More Regularizations on Less Robust Samples (ICML 2023 · similarity 0.89)