Research map / Adversarial robustness and security / Black-box and transfer attacks
Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Forward Propagation Refinement (FPR), combining Attention Map Diversification and Momentum Token Embedding to refine ViT surrogates during the forward pass to boost transferable adversarial examples.
Problem
Improving the transferability of adversarial examples crafted on Vision Transformer surrogate models, where prior surrogate refinements only modify backward propagation.
Approach
Refine ViT surrogates in the forward pass rather than the backward pass: randomly diversify selected attention maps (AMD, which also implicitly induces gradient vanishing) and accumulate historical token embeddings with momentum (MTE), both integrated into MIM.
Key results
- With ViT-B as surrogate, FPR reaches 84.3% average attack success rate on six CNNs, beating GNS, TGR and PNAPO by 7.0%, 7.4% and 15.5%.
- Against five defenses with ViT-B surrogate, FPR achieves 70.8% average ASR, exceeding GNS by 5.7%.
- Combined with VTM on CNN targets, FPR raises ASR from 57.3% to 81.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FPR (AMD + MTE) to MIM | ViT-B surrogate attack · 13 target models (excluding ViT-B) | ASR (%): 77.3 -> 84.3 | Helped |
| Add FPR (AMD + MTE) to MIM | CaiT-S surrogate attack · 13 target models (excluding CaiT-S) | ASR (%): 85.3 -> 88.1 | Helped |
| Add FPR (AMD + MTE) to MIM | PiT-T surrogate attack · 13 target models (excluding PiT-T) | ASR (%): 66.6 -> 72.9 | Helped |
| Add FPR (AMD + MTE) to MIM | DeiT-B surrogate attack · 13 target models (excluding DeiT-B) | ASR (%): 87.2 -> 90.2 | Helped |
| Add FPR (AMD + MTE) to MIM | ViT-B surrogate attack · Five defenses | ASR (%): 65.1 -> 70.8 | Helped |
| Add FPR to SIA | ViT-B surrogate attack · CNNs | ASR (%): 69.1 -> 90.4 | Helped |
| Add FPR to BSR | ViT-B surrogate attack · CNNs | ASR (%): 66.9 -> 89.9 | Helped |
| Add FPR to VTM | ViT-B surrogate attack · CNNs | ASR (%): 57.3 -> 81.7 | Helped |
| Add FPR to GRA | ViT-B surrogate attack · CNNs | ASR (%): 68.3 -> 85.3 | Helped |
| Add AMD to MIM (ablation) | ViT-B surrogate attack · ViTs | ASR (%): 51.6 -> 82.1 | Helped |
| Add AMD to MIM (ablation) | ViT-B surrogate attack · CNNs | ASR (%): 50.2 -> 78.4 | Helped |
| Add MTE to MIM (ablation) | ViT-B surrogate attack · ViTs | ASR (%): 51.6 -> 78.4 | Helped |
Surprising findings
- Randomly dropping a small number of attention blocks in the surrogate improves transferability, indicating redundancy-driven overfitting.
- Dropout on attention maps is much less effective than AMD, since removing attention maps discards too much information.
Limitations
- Hyperparameters (index set, diversity, scale, attenuation factors) are tuned per surrogate model.
- Evaluated only on 1000 ImageNet validation images and ViT surrogates with MIM as base attack.
Experimental setup
Datasets: ILSVRC2012 validation set (1000 images)
Benchmarks: ILSVRC2012 validation subset (attack success rate on 13 ViTs/CNNs) five defenses: AT, HGD, NRP, JPEG, Bit depth reduction Baidu Cloud API Aliyun API
Metrics: attack success rate (%)
Baselines: MIM PNAPO TGR GNS Dropout VTM GRA BSR SIA
Builds on: MIM PNAPO TGR Momentum Distillation
Closest papers by meaning
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.90)
- Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability (NeurIPS 2025 · similarity 0.90)
- On the Surprising Effectiveness of Attention Transfer for Vision Transformers (NeurIPS 2024 · similarity 0.89)
- Improving the Adversarial Transferability of Vision Transformers with Virtual Dense Connection (AAAI 2024 · similarity 0.89)
- Revisiting Continuity of Image Tokens for Cross-domain Few-shot Learning (ICML 2025 · similarity 0.87)
- LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.