Research map / Adversarial robustness and security / Black-box and transfer attacks
Ensemble Diversity Facilitates Adversarial Transferability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMER, an ensemble adversarial attack that perturbs each surrogate model individually in stochastic mini-batches and reweights ensemble weights with an RL-style reward, improving transferability especially to heterogeneous (CNN-to-ViT) victims.
Problem
Transfer-based black-box attacks using model ensembles; existing ensemble methods (SVRE, AdaEA) reduce perturbation diversity across surrogates, which hurts transferability especially for heterogeneous CNN/ViT attacks and mixed ensembles.
Approach
Instead of suppressing diversity, treat each surrogate as an individual: generate perturbations from one randomly selected surrogate per inner step (stochastic mini-batch perturbing), and learn ensemble weights via a reinforcement-learning-style update that maximizes attack loss (ensemble reweighing), plugged into I-FGSM/MI-FGSM/TIM/TI-DIM.
Key results
- On adversarial-trained victims with TI-DIM, SMER reaches 96.3% average ASR vs 82.2% for SVRE... (per Table 1 average) and 83.3% average on robust ViTs.
- Targeted attacks on TI-DIM: SMER average ASR 27.3% vs 5.1% for SVRE on the adversarial-trained models.
- Ensemble types: SMER achieves 98.3% average ASR on ViT victims when ensembled with only ViTs, 10.3% above AdaEA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SMER instead of the strongest compared ensemble attack | Black-box attacks on CNN and ViT victims · CNN and ViT victims (Table 2 averages) | ASR (%) | Helped |
| Use SMER rather than SVRE for targeted TI-DIM attacks | Targeted attack on adversarial-trained victims · Inc-v3ens3, Inc-v3ens4, IncRes-v2ens (average) | ASR (%): 27.3 vs 5.1 | Helped |
| Use random rather than sequential model selection (ablation) | SMER · Adversarial-trained victims | ASR (%): about 5% | Helped |
| Add ensemble reweighing (ablation) | Base ensemble attack · Adversarial-trained victims | ASR (%) | Helped |
| Add stochastic mini-batch perturbing (ablation) | Base ensemble attack with reweighing · Adversarial-trained victims | ASR (%) | Helped |
Surprising findings
- Existing diversity-reducing ensemble methods (SVRE, AdaEA) underperform a plain logit ensemble on several heterogeneous settings, as shown by ASR drops in mixed ensembles.
- Constraining gradient diversity across surrogates appears incompatible with targeted attacks, per the authors' reading of their results.
Limitations
- Gains are reported mainly against ImageNet-compatible models and a limited set of defenses.
- Reweighing uses an SGD learning rate of 0.02 and a simplified RL formulation; sensitivity to these choices is not deeply analyzed.
- Paper states that cosine-based discrepancy rises for baselines, but the explanation is heuristic.
Experimental setup
Datasets: ImageNet-compatible dataset
Benchmarks: ImageNet-compatible adversarial training defenses (Inc-v3ens3, Inc-v3ens4, IncRes-v2ens) transformation defenses (R&P, NIPS-r3, Bit-R, JPEG, FD, ComDefend) robust ViTs (RVT-S*, Drvit, ViT+DAT) CNN and ViT victims (Res-50, WRN101, Inc-v4, BiT, ViT, DeiT, Swin)
Metrics: attack success rate (%) average loss SSIM attention alteration gradient discrepancy
Baselines: Ens (logit ensemble) SVRE AdaEA I-FGSM MI-FGSM TIM TI-DIM
Builds on: MI-FGSM SVRE AdaEA TI-DIM
Closest papers by meaning
- Understanding Model Ensemble in Transferable Adversarial Attack (ICML 2025 · similarity 0.91)
- Enhancing Adversarial Transferability with Checkpoints of a Single Model's Training (CVPR 2025 · similarity 0.91)
- Rethinking Model Ensemble in Transfer-based Adversarial Attacks (ICLR 2024 · similarity 0.90)
- Boosting Adversarial Transferability through Augmentation in Hypothesis Space (CVPR 2025 · similarity 0.88)
- Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning (CVPR 2024 · similarity 0.88)
- LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.