Research map / Adversarial robustness and security / Black-box and transfer attacks
Learning to Transform Dynamically for Better Adversarial Transferability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Learning to Transform (L2T), which dynamically samples and updates a probability distribution over input transformation operations during iterative adversarial attacks to boost transferability.
Problem
Input transformation-based transfer attacks use fixed or limited transformations, and the search space for the optimal per-iteration transformation combination is exponentially large, so transferability is limited.
Approach
Cast selection of per-iteration transformation combinations as an optimal trajectory search; sample transformation combinations from a learnable categorical distribution over operations and update that distribution by gradient ascent on the attack loss within each iteration, without extra training modules.
Key results
- L2T outperforms the strongest baseline by 22.9% average ASR across single surrogate models (ten target models).
- On defenses, L2T achieves 47.9% (AT), 98.5% (HGD), 87.2% (NRP), and 46.7% (RS) ASR with ensemble attacks.
- L2T achieves nearly 100% attack success rate on GPT-4V and Gemini (Bard) and beats the strongest baseline by 8.7% and 12.6% on Google Vision and Azure AI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use L2T instead of the strongest baseline | Ensemble attack · RS | ASR: 46.7%; exceeds the best baseline (AITL) by 4.6% | Helped |
| Use L2T instead of the strongest baseline | Ensemble attack · Google Vision | ASR: 8.7% | Helped |
| Use L2T instead of the strongest baseline | Ensemble attack · Azure AI | ASR: 12.6% | Helped |
| Use L2T instead of the strongest baseline | Ensemble attack · GPT-4V and Gemini | ASR: nearly 100% | Helped |
| Increase the number of operations from 1 to 2 (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 80.89% to 88.98% | Helped |
| Increase the number of operations from 2 to 5 (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 2.29% | Helped |
| Increase the number of sampled transformations from 1 to 20 (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 75.7% to 91.1% | Helped |
| Increase the number of sampled transformations beyond 20 (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 1.5% | Helped |
| Use gradient-guided sampling instead of random transformation sampling (ablation) | L2T · ImageNet validation | ASR: minimum difference is 31.12% | Helped |
| Add Scale to the transformation candidates (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 76.2 -> 89.7 | Helped |
| Add Rotate to the transformation candidates (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 78.8 -> 89.7 | Helped |
| Add Mixup to the transformation candidates (ablation) | L2T · ImageNet validation, targets excluding ResNet-18 | average ASR: 86.6 -> 89.7 | Helped |
Surprising findings
- Combining more transformations does not consistently help; some transformation trajectories perform worst despite more diversity.
- Random transformation sampling performs far worse than gradient-guided sampling (gap of at least 31%).
Limitations
- Gains on the RS certified defense are modest (46.7% ASR).
- Hyperparameters such as number of operations K and number of transformations L are chosen via ablations on ResNet-18 only.
Experimental setup
Datasets: ImageNet (ILSVRC 2012 validation subset)
Benchmarks: ImageNet ILSVRC 2012 validation subset (1,000 images) Google Vision Azure AI GPT-4V Bard
Metrics: attack success rate (ASR)
Baselines: TIM SIM Admix DEM IDE Mask S2IM BSR SIA AutoMA ATTA AITL
Builds on: MI-FGSM I-FGSM Admix DIM AutoMA AITL
Closest papers by meaning
- Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability (NeurIPS 2025 · similarity 0.90)
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.89)
- Boosting Adversarial Transferability through Augmentation in Hypothesis Space (CVPR 2025 · similarity 0.88)
- Enhancing Adversarial Transferability with Adversarial Weight Tuning (AAAI 2025 · similarity 0.88)
- Improving the Adversarial Transferability of Vision Transformers with Virtual Dense Connection (AAAI 2024 · similarity 0.87)
- Boosting Adversarial Transferability by Block Shuffle and Rotation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.