Research map / Adversarial robustness and security / Certified and empirical robustness
AUTE: Peer-Alignment and Self-Unlearning Boost Adversarial Robustness for Training Ensemble Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AUTE, which combines adversarial training with ensemble learning via peer-alignment (round-robin KL alignment of each member to a frozen peer ensemble) and self-unlearning (down-weighted CE on over-confident examples), improving robust accuracy of ResNet-20 ensembles on CIFAR-10 and other datasets.
Problem
Simple combinations of adversarial training and ensemble learning can fail to improve, and sometimes degrade, adversarial robustness of ensemble models.
Approach
Partition the ensemble into a frozen large peer model and one optimized member, align the member to the peer via selectively weighted KL divergence, and apply a self-unlearning loss that down-weights over-confident (large-margin) examples; iterate in round-robin over members.
Key results
- A 3-member AUTE with ResNet-20 reaches 54.6% average robust accuracy on CIFAR-10 versus 51.6% for the SOTA baseline.
- An 8-member AUTE reaches 56.8% average robustness on CIFAR-10 versus 53.0% for the best baseline.
- On CIFAR-100 AUTE outperforms AROW by 0.9% clean accuracy and 0.6% average robustness.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Peer-Alignment (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | clean accuracy (%): 76.7 -> 79.0 | Helped |
| add Peer-Alignment (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 54.4 -> 54.8 | Helped |
| add Self-Unlearning (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 50.3 -> 54.8 | Helped |
| remove sample-selective alignment weights (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 54.4 -> 54.8 | Hurt |
| replace Peer-Alignment with ADP diversification (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 45.0 -> 54.8 | Hurt |
| replace Self-Unlearning with label smoothing (λ=0.1) (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 48.5 -> 54.8 | Hurt |
| replace Self-Unlearning with label smoothing (λ=0.3) (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 49.0 -> 54.8 | Hurt |
| replace Self-Unlearning with label smoothing (λ=0.5) (ablation) | AUTE ensemble · CIFAR-10, 3-member ResNet-20 | Avg robust accuracy (%): 49.2 -> 54.8 | Hurt |
| increase label-smoothing coefficient (ablation) | AUTE-LS · CIFAR-10, 3-member ResNet-20 | clean accuracy (%): 79.2 -> 79.1 -> 78.6 | Hurt |
Surprising findings
- Simply combining adversarial training with ensemble learning can marginally improve or even degrade adversarial robustness.
- Diversity regularization (ADP) reduces robustness of adversarially trained ensembles.
Limitations
- Gains over strongest baselines are modest on some datasets (e.g., Tiny-ImageNet average robustness +0.5%).
- Robustness shows bias trade-offs against clean accuracy depending on network family; VGG gains in robustness are limited.
Experimental setup
Datasets: CIFAR-10 MNIST CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 MNIST CIFAR-100 Tiny-ImageNet
Metrics: clean accuracy robust accuracy
Baselines: TRADES MART HAT STAT MCE AROW ADP GAL DVERGE TRS
Builds on: TRADES MART ADP DVERGE MCE
Closest papers by meaning
- Improving Ensemble Robustness by Collaboratively Promoting and Demoting Adversarial Robustness (AAAI 2021 · similarity 0.89)
- MORA: Improving Ensemble Robustness Evaluation with Model Reweighing Attack (NeurIPS 2022 · similarity 0.88)
- PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor (CVPR 2024 · similarity 0.86)
- Understanding Model Ensemble in Transferable Adversarial Attack (ICML 2025 · similarity 0.85)
- Rethinking Model Ensemble in Transfer-based Adversarial Attacks (ICLR 2024 · similarity 0.85)
- Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.