Research map / Adversarial robustness and security / Certified and empirical robustness
TAET: Two-Stage Adversarial Equalization Training on Long-Tailed Distributions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TAET, a two-stage adversarial training method for long-tailed data (cross-entropy warm-up, then hierarchical equalization adversarial training with BCL/HDL/RCEL losses), and a Balanced Robustness metric.
Problem
Adversarial robustness of deep networks degrades under long-tailed class distributions, where tail and weak classes are poorly robust and robust overfitting occurs; existing metrics ignore balanced accuracy/robustness.
Approach
A cross-entropy warm-up stage followed by an adversarial phase whose loss combines balanced cross-class loss, hierarchical deviation loss (variance of class losses) and rare class emphasis loss, plus a balanced robustness metric defined as class-averaged robust recall.
Key results
- On CIFAR-10-LT (IR=10, ResNet-18), TAET reaches 30.57% balanced robustness under AutoAttack versus 25.26% for AT-BSL, a 5.31-point gain.
- On CIFAR-10-LT tail classes under PGD-20, TAET achieves higher robust accuracy than AT-BSL on several classes (e.g., Frog 34.83 vs 26.97).
- On DermaMNIST, TAET attains 40.12% balanced robustness under FGSM and 21.74% under AutoAttack, with clean balanced accuracy of 48.42% close to AT-BSL's 48.55%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace AT-BSL with TAET | ResNet-18 · CIFAR-10-LT (IR=10), AutoAttack | balanced robustness: 25.26 -> 30.57; 5.31% improvement | Helped |
| Replace AT-BSL with TAET | ResNet-18 · CIFAR-10-LT (IR=10), PGD-20 | balanced robustness: 26.86 -> 35.15 | Helped |
| Replace AT-BSL with TAET | ResNet-18 · DermaMNIST, PGD-20 | balanced robustness: 27.85 -> 29.26 | Helped |
| Replace AT-BSL with TAET | ResNet-18 · DermaMNIST, balanced accuracy | balanced accuracy: 48.55 -> 48.42 | Hurt |
| Use cross-entropy loss for 40 initial epochs instead of omitting it (ablation) | TAET · CIFAR-10-LT (IR=10) | clean sample accuracy: +7.19% | Helped |
| Use cross-entropy loss for 40 initial epochs instead of omitting it (ablation) | TAET · CIFAR-10-LT (IR=10), AutoAttack | balanced robustness: +1.04% | Helped |
| Add BCL (ablation) | HARL · CIFAR-10-LT | accuracy | Helped |
| Add HDL and RCEL together (ablation) | HARL · CIFAR-10-LT | performance on weak classes | Helped |
| Set equal loss weights α = β = γ (ablation) | TAET · CIFAR-10-LT (IR=10) | balanced robustness: 35.15 | Helped |
Surprising findings
- TAET's clean balanced accuracy on DermaMNIST (48.42%) is slightly below AT-BSL (48.55%).
Limitations
- Very rare classes (e.g., Class 3 and Class 6 on DermaMNIST) still show high variability and may need larger loss weights.
- Hyperparameter sensitivity: alpha, beta, gamma must be balanced; squared loss terms may affect training stability.
- Evaluation limited to small benchmarks (CIFAR-10-LT, CIFAR-100-LT, DermaMNIST) with ResNet-18 and ViT-B/16.
Experimental setup
Datasets: CIFAR-10-LT CIFAR-100-LT DermaMNIST
Benchmarks: CIFAR-10-LT CIFAR-100-LT DermaMNIST
Metrics: balanced accuracy balanced robustness clean accuracy robust accuracy under FGSM, PGD-20, PGD-100, CW, AutoAttack
Baselines: AT-BSL RoBal TRADES AT MART ADT REAT LAS-AT HE GAIRAT
Builds on: AT-BSL PGD adversarial training RoBal TRADES Balanced Softmax Loss
Closest papers by meaning
- Long-tailed Adversarial Training with Self-Distillation (ICLR 2025 · similarity 0.92)
- Revisiting Adversarial Training Under Long-Tailed Distributions (CVPR 2024 · similarity 0.92)
- Boosting Adversarial Robustness with CLAT: Criticality Leveraged Adversarial Training (ICML 2025 · similarity 0.88)
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.88)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.88)
- Stability and Generalization of Adversarial Training for Shallow Neural Networks with Smooth Activation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.