Research map / Adversarial robustness and security / Certified and empirical robustness

Revisiting Adversarial Training Under Long-Tailed Distributions

CVPR 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training refinements · 18 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that balanced softmax loss alone matches the RoBal adversarial training method on long-tailed CIFAR-10 at lower cost, and that data augmentation (notably RandAugment and AutoAugment) mitigates robust overfitting while improving robustness via increased training data diversity.

Problem

Adversarial training has mostly been evaluated on balanced datasets, but real-world data are long-tailed; the only prior long-tailed adversarial method (RoBal) is complex and costly, and robust overfitting under long-tailed distributions is unexplored.

Approach

Ablating RoBal reveals Balanced Softmax Loss as its key component, so AT-BSL is a cheaper substitute; and unlike balanced data, data augmentation under long-tailed data both reduces robust overfitting and improves robustness, attributed to increased training diversity.

Key results

• On CIFAR-10-LT with WideResNet-34-10, AT-BSL with AutoAugment reaches +6.66% AutoAttack robustness over RoBal (best checkpoint).
• AT-BSL alone (ResNet-18, CIFAR-10-LT) reaches 70.21% clean accuracy and 29.48% AA-style robustness vs. RoBal 70.34% and 29.54%, with training time 946 s/epoch vs 1722 s/epoch and memory 946 vs 1722 MiB (Table 1).
• Data augmentation such as RandAugment raises AT-BSL AutoAttack robustness from 31.78% to 34.24% on ResNet-18 CIFAR-10-LT at the best checkpoint, and reduces robust overfitting at the final checkpoint.

Measured effects reported in the paper

Surprising findings

• Data augmentation alone improves adversarial robustness under long-tailed distributions, unlike prior findings on balanced data where it does not.
• The RoBal public code's learning-rate schedule (decaying to about 1e-26) differs from the paper's description, which suppressed robust overfitting in the code's results.

Limitations

• Main experiments are on CIFAR-10-LT and CIFAR-100-LT only, with a fixed imbalance ratio of 50 in the main tables.
• AT-BSL is sensitive to the hyperparameter tau_b on CIFAR-10-LT (optimal at 1).
• Using augmentation with reduced PGD step size lowers robustness relative to the 2/255 baseline.

Experimental setup

Datasets: CIFAR-10-LT CIFAR-100-LT CIFAR-10 CIFAR-100 Tiny-ImageNet

Benchmarks: CIFAR-10-LT CIFAR-100-LT

Metrics: clean accuracy robust accuracy under FGSM PGD-20 CW LSA AutoAttack training time per epoch GPU memory

Baselines: AT TRADES MART AWP GAIRAT LAS-AT RoBal REAT

Builds on: AT RoBal Balanced Softmax Loss TRADES RandAugment

Closest papers by meaning

• Adversarial Robustness Under Long-Tailed Distribution (CVPR 2021 · similarity 0.93)
• Long-tailed Adversarial Training with Self-Distillation (ICLR 2025 · similarity 0.92)
• TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributions (CVPR 2025 · similarity 0.92)
• Understanding Robust Overfitting of Adversarial Training and Beyond (ICML 2022 · similarity 0.90)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.88)
• Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.87)