Research map / Adversarial robustness and security / Certified and empirical robustness
Are Vision Transformers Robust to Patch Perturbations?
Jindong Gu, Volker Tresp, Yao Qin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ResNet50 with DeiT-small (same training recipe), 196 naturally corrupted patches | ViT vs CNN · ImageNet-1k val | fooling rate (%): 49.8 -> 38.9 | Improved |
| replace ResNet50 with DeiT-small, 1 adversarial patch | ViT vs CNN · ImageNet-1k val | fooling rate (%): 30.6 -> 61.5 | Hurt |
| replace ResNet18 with DeiT-tiny, 1 adversarial patch | ViT vs CNN · ImageNet-1k val | fooling rate (%): 39.4 -> 63.3 | Hurt |
| add Smoothed Attention (softmax temperature T>1) (ablation) | DeiT · ImageNet-1k val adversarial patch | fooling rate | Improved |
| add Smoothed Attention to Swin Transformer | Swin · ImageNet-1k val adversarial patch | fooling rate | No clear effect |
| add Weight Standardization (ablation) | ResNet18 · CIFAR10 single patch attack | fooling rate (%): 35.9 -> 51.3 | Hurt |
| add Group Normalization (ablation) | ResNet18 · CIFAR10 single patch attack | fooling rate (%): 35.9 -> 52.6 | Hurt |
| add Weight Standardization and Group Normalization (ablation) | ResNet18 · CIFAR10 single patch attack | fooling rate (%): 35.9 -> 71.1 | Hurt |
| increase input size from 32 to 48 (ablation) | ResNet18 · CIFAR10 single patch attack | fooling rate (%): 35.9 -> 42.2 | No clear effect |
| align adversarial patch shift to patch grid (16 px vs 1 px) (ablation) | DeiT-small · ImageNet-1k val patch transferability | transfer fooling rate: 0.27 -> 8.43 | Improved |
| use ViT-agnostic (unaligned) adversarial patch instead of aligned (ablation) | DeiT-small · ImageNet-1k val adversarial patch | fooling rate (%): 61.5 -> 47.9 | Hurt |
| replace self-attention with MLP-Mixer token mixing | ViT · ImageNet-1k adversarial patch | fooling rate (%): 31.36 | Improved |
Closest papers by meaning
- Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations? (ICLR 2022 · similarity 0.94)
- Understanding and Defending Patched-based Adversarial Attacks for Vision Transformer (ICML 2023 · similarity 0.93)
- Improving Interpretation Faithfulness for Vision Transformers (ICML 2024 · similarity 0.90)
- Attacking Transformers with Feature Diversity Adversarial Perturbation (AAAI 2024 · similarity 0.90)
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.89)
- Towards Understanding and Improving Adversarial Robustness of Vision Transformers (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
