Research map / Adversarial robustness and security / Certified and empirical robustness
On the Diversity of Adversarial Ensemble Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that precisely computing adversarial diversity between two networks is NP-hard, derives a first-order diversity decomposition for adversarial ensemble loss (squared and cross-entropy), and proposes AdvEOAP, an adversarial ensemble training method using orthogonal adversarial predictions.
Problem
Defining and measuring diversity for adversarial ensemble learning, and understanding how diversity correlates with adversarial robustness; prior diversity measures (cos of gradients, prediction disagreement) are incomplete.
Approach
Proves NP-hardness of exact adversarial diversity and decomposes the first-order adversarial ensemble loss into average individual adversarial loss, prediction diversity, gradient diversity and cross diversity; then regularizes base learners to have orthogonal adversarial output predictions (log-det volume plus entropy) to raise gradient and cross diversity jointly.
Key results
- AdvEOAP reaches 95.68% on MNIST under PGD10 and 53.08% on CIFAR10 under PGD10 with l-inf perturbation, outperforming baselines such as AdvADP and TRS on most attacks.
- Under EOT attack on CIFAR10, AdvEOAP achieves 42.00% versus 40.79% for AdvADP and 31.63% for TRS.
- Removing the orthogonality regularization lowers APGD accuracy by about 1.2-1.4 points across MNIST, F-MNIST and CIFAR10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · MNIST, PGD20 | accuracy: 95.059±0.089 -> 95.648±0.056 | Helped |
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · MNIST, AutoAttack | accuracy: 92.948±0.062 -> 94.072±0.423 | Helped |
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · F-MNIST, PGD20 | accuracy: 79.074±0.387 -> 81.752±0.248 | Helped |
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · F-MNIST, AutoAttack | accuracy: 77.732±0.245 -> 80.506±0.327 | Helped |
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · CIFAR10, PGD20 | accuracy: 46.802±0.284 -> 52.996±0.255 | Helped |
| Use AdvEOAP instead of AdvADP | Adversarial ensemble with ResNet20 · CIFAR10, AutoAttack | accuracy: 42.174±0.198 -> 47.884±0.060 | Helped |
| Add regularization Γα(·) (ablation) | AdvEOAP · MNIST, APGD | accuracy: 93.513±0.046 -> 94.907±0.187 | Helped |
| Add regularization Γα(·) (ablation) | AdvEOAP · F-MNIST, APGD | accuracy: 80.317±0.159 -> 81.202±0.311 | Helped |
| Add regularization Γα(·) (ablation) | AdvEOAP · CIFAR10, APGD | accuracy: 50.833±0.102 -> 51.997±0.234 | Helped |
Surprising findings
- Adversarial ensemble loss keeps decreasing with more base learners while average cos value and average individual adversarial loss stay nearly constant, so cos-based diversity alone is insufficient.
Limitations
- Diversity decomposition relies on first-order Taylor approximation; second-order analysis left open.
- Method cost scales with number of base learners (m times adversarial training, O(m^3) regularizer cost).
- Experiments limited to three small image datasets with ResNet20 base learners.
Experimental setup
Datasets: MNIST F-MNIST CIFAR10
Benchmarks: MNIST F-MNIST CIFAR10
Metrics: adversarial accuracy (mean ± std over 50 runs) paired t-test significance
Baselines: GAL ADP AdvADP DVERGE PDD TRS iGATADP
Builds on: error-ambiguity decomposition ADP (orthogonal predictions ensemble) GAL (gradient diversity) PGD adversarial training
Closest papers by meaning
- Improving Adversarial Robustness via Promoting Ensemble Diversity (ICML 2019 · similarity 0.93)
- Adversarial Defence by Diversified Simultaneous Training of Deep Ensembles (AAAI 2021 · similarity 0.90)
- Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance (NeurIPS 2024 · similarity 0.86)
- Ensemble Diversity Facilitates Adversarial Transferability (CVPR 2024 · similarity 0.86)
- Understanding Model Ensemble in Transferable Adversarial Attack (ICML 2025 · similarity 0.86)
- Rethinking Model Ensemble in Transfer-based Adversarial Attacks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.