Research map / Adversarial robustness and security / Black-box and transfer attacks
Boosting Adversarial Transferability by Block Shuffle and Rotation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes block shuffle and rotation (BSR), an input-transformation transfer attack that shuffles and rotates image blocks to diversify gradients and stabilize attention heatmaps across models, improving black-box transferability on ImageNet.
Problem
Adversarial examples crafted on a single white-box surrogate transfer poorly to other models; existing input transformation attacks produce inconsistent attention heatmaps across models, limiting transferability.
Approach
Split the image into n×n blocks, randomly shuffle and rotate each block to break intrinsic image structure, and average gradients over N such transformed images inside MI-FGSM, so the perturbation is optimized to make attention heatmaps consistent across models.
Key results
- BSR achieves 93.8% average attack success rate on standard-trained black-box models from Inc-v3 source, at least 6.5% above Admix.
- On adversarially trained models, BSR reaches 56.2% average success, 11.0% above PAM.
- BSR-SI-TI-DIM reaches 94.1% average success against eight defenses, outperforming PAM-TI-DIM by 2.5% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply BSR instead of Admix | MI-FGSM attacks from Inc-v3 · ImageNet, Inc-v3 source → Inc-v4 | attack success rate: 82.2 -> 96.2 | Helped |
| Apply BSR instead of Admix | MI-FGSM attacks from Inc-v3 · ImageNet, Inc-v3 source → IncRes-v2 | attack success rate: 81.1 -> 94.7 | Helped |
| Apply BSR instead of Admix | MI-FGSM attacks from Inc-v3 · ImageNet, Inc-v3 source → Res-101 | attack success rate: 73.8 -> 90.5 | Helped |
| Combine BSR with SI-TI-DIM instead of using Admix-TI-DIM | Ensemble attack · ImageNet, adversarially trained models | attack success rate: 95.8 -> 99.1 | Helped |
| Combine BSR with SI-TI-DIM instead of using PAM-TI-DIM | Ensemble attack · ImageNet, defended models (average) | attack success rate: 91.6 -> 94.1 | Helped |
| Add block shuffling (ablation) | MI-FGSM · ImageNet, black-box transfer | attack success rate | Helped |
| Add block rotation (ablation) | MI-FGSM · ImageNet, black-box transfer | attack success rate | Helped |
| Combine block shuffling and block rotation (ablation) | MI-FGSM · ImageNet, black-box transfer | attack success rate | Helped |
| Use one block (rotation without block shuffling) (ablation) | BSR · ImageNet, black-box transfer | attack success rate | Hurt |
| Increase the number of blocks beyond 3 (ablation) | BSR · ImageNet, black-box transfer | attack success rate | Hurt |
| Increase transformed images to more than 20 (ablation) | BSR · ImageNet, black-box transfer | attack success rate | No clear effect |
| Increase maximum rotation angle below 24° (ablation) | BSR · ImageNet, black-box transfer | attack success rate | Helped |
Surprising findings
- Block shuffling alone breaks attention heatmaps, yet shuffled blocks can still be correctly classified about 86.8% of the time on Inc-v3.
Limitations
- Gains on some adversarially trained models are smaller in combined settings, and BSR-TI-DIM underperforms on IncRes-v2ens relative to some baselines.
- Computing gradients over N=20 transformed images per iteration adds cost relative to single-image attacks.
- Evaluated only on ImageNet with 1000 validation images and ℓ∞ budget 16.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet
Metrics: attack success rate
Baselines: DIM TIM SIM Admix PAM MI-FGSM TI-DIM SI-TI-DIM
Builds on: MI-FGSM DIM TIM SIM Admix PAM RLFAT
Closest papers by meaning
- Rethinking the Backward Propagation for Adversarial Transferability (NeurIPS 2023 · similarity 0.88)
- Learning to Transform Dynamically for Better Adversarial Transferability (CVPR 2024 · similarity 0.87)
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.87)
- Enhancing Adversarial Transferability with Adversarial Weight Tuning (AAAI 2025 · similarity 0.87)
- Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability (NeurIPS 2025 · similarity 0.86)
- Boosting Adversarial Transferability through Augmentation in Hypothesis Space (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.