Research map / Adversarial robustness and security / Certified and empirical robustness
Improving the Adversarial Transferability of Vision Transformers with Virtual Dense Connection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Virtual Dense Connection (VDC), which recomposes ViTs with virtual dense skip connections so that gradients from deeper blocks are back-propagated to craft more transferable adversarial examples, without changing the forward pass.
Problem
Adversarial examples crafted on vision transformers transfer poorly, both between ViT architectures and especially from ViTs to CNNs (cross-structure transferability).
Approach
A motivating study with gradient down-scaling shows skip-connection gradients matter most for transferability; VDC recomposes the network with virtual dense connections (keeping the forward pass) and back-propagates scaled gradients of Attention maps and MLP blocks from deeper blocks, approximated in O(1) per block.
Key results
- VDC reaches 100% white-box attack success on ViT-B/16 and beats the best baseline by about 8.2% on average in transformer-to-transformer black-box transfer.
- VDC improves cross-structure transfer from ViTs to CNNs by over 7.2% on average over baselines.
- Ablation on ViT-B/16: dense connection on Attention plus MLP (VDC) gives 75.1% ViT, 44.5% CNN, 31.8% adversarially trained CNN average transferability, versus 56.2/29.0/19.5 with none.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VDC instead of the strongest baseline | ViT-B/16 transfer attacks · Transformer-based models | average attack success rate: 8.2% | Helped |
| use VDC instead of the strongest baseline | ViT-B/16 transfer attacks · ViT-to-CNN transfer | average attack success rate: 7.2% | Helped |
| add dense connections to MLP blocks (ablation) | VDC on ViT-B/16 · ViTs | adversarial transferability: 56.2 -> 66.0 | Helped |
| add dense connections to MLP blocks (ablation) | VDC on ViT-B/16 · CNNs | adversarial transferability: 29.0 -> 34.4 | Helped |
| add dense connections to MLP blocks (ablation) | VDC on ViT-B/16 · Adversarially trained CNNs | adversarial transferability: 19.5 -> 24.1 | Helped |
| add dense connections to Attention maps (ablation) | VDC on ViT-B/16 · ViTs | adversarial transferability: 56.2 -> 66.8 | Helped |
| add dense connections to Attention maps (ablation) | VDC on ViT-B/16 · CNNs | adversarial transferability: 29.0 -> 37.0 | Helped |
| add dense connections to Attention maps (ablation) | VDC on ViT-B/16 · Adversarially trained CNNs | adversarial transferability: 19.5 -> 24.6 | Helped |
| add dense connections to both Attention maps and MLP blocks (ablation) | VDC on ViT-B/16 · ViTs | adversarial transferability: 56.2 -> 75.1 | Helped |
| add dense connections to both Attention maps and MLP blocks (ablation) | VDC on ViT-B/16 · CNNs | adversarial transferability: 29.0 -> 44.5 | Helped |
| add dense connections to both Attention maps and MLP blocks (ablation) | VDC on ViT-B/16 · Adversarially trained CNNs | adversarial transferability: 19.5 -> 31.8 | Helped |
| down-scale the gradient through an Attention-block skip connection (ablation) | ViT-B/16 · ViTs, CNNs, and adversarially trained CNNs | adversarial transferability | Hurt |
Surprising findings
- Down-scaling the gradient of skip connections causes the largest drop in transferability, more than scaling attention or MLP layer gradients.
Limitations
- Relies on a one-consecutive-skip approximation of the full gradient expansion.
- Cross-structure transfer to CNNs remains much lower than transformer-to-transformer transfer.
- Hyperparameters (scaling factor 0.1, decay 0.5) were set following baselines and are not extensively tuned.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet
Metrics: attack success rate
Baselines: MIM SGM SE PNA
Builds on: MIM SGM Self-Ensemble PNA
Closest papers by meaning
- Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability (NeurIPS 2025 · similarity 0.92)
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.91)
- Attacking Transformers with Feature Diversity Adversarial Perturbation (AAAI 2024 · similarity 0.89)
- Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement (CVPR 2025 · similarity 0.89)
- Random Entangled Tokens for Adversarially Robust Vision Transformer (CVPR 2024 · similarity 0.88)
- Enhancing Adversarial Transferability with Adversarial Weight Tuning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.