Research map / Adversarial robustness and security / Certified and empirical robustness

Random Entangled Tokens for Adversarially Robust Vision Transformer

CVPR 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReiT, a randomized adversarial-robustness defense for ViTs that concatenates random tokens into self-attention (II-ReSA, with a second optimized "entangled" token) combined with adversarial training.

Problem

Adversarial vulnerability of Vision Transformers; standard randomization defenses designed for CNNs (e.g., randomized normalization) do not transfer directly to ViT architectures.

Approach

Concatenate a random token to the ViT input and replace global softmax with a local softmax so the random token adds an additive random term to self-attention; at test time, optimize a second random "entangled" token via gradient sign steps to make gradients dissimilar, then randomly sample one of the two tokens per inference.

Key results

• On CIFAR-10 with DeiT-T under PGD adversarial training, ReiT raises PGD-attack accuracy from 49.33% to 50.07% and DeepFool accuracy from 1.04% to 26.25%, with a clean-accuracy drop from 82.03% to 80.73%.
• For ViT-S under PGD adversarial training on CIFAR-10, ReiT achieves 52.86% under PGD vs 51.51% for the vanilla baseline (+1.35%).
• Compared with ARD & PRM on CIFAR-10, ReiT gives 50.39% AutoAttack accuracy for DeiT-T vs 46.83% for ARD & PRM (vanilla 45.54%).

Measured effects reported in the paper

Surprising findings

• Using a random token without optimizing the second entangled token gives no significant robustness improvement, according to the component ablation.
• Too large randomness intensity (0.4) substantially degrades both clean and adversarial accuracy.

Limitations

• Clean accuracy drops relative to vanilla adversarially trained models.
• Experiments limited to small and tiny ViT variants due to compute constraints.
• Robustness gains on PGD are modest (e.g., +1.35% for ViT-S) and some gains are attributed to randomness rather than proven.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 ImageNette ImageNet-1K

Benchmarks: CIFAR-10 CIFAR-100 ImageNette ImageNet-1K

Metrics: clean accuracy adversarial accuracy under NAT, FGSM, MIFGSM, PGD, DeepFool,

Baselines: PGD adversarial training FGSM adversarial training TRADES ARD & PRM NAT

Builds on: PGD adversarial training TRADES Vision Transformer DeiT Swin Transformer

Closest papers by meaning

• Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.89)
• Improving the Adversarial Transferability of Vision Transformers with Virtual Dense Connection (AAAI 2024 · similarity 0.88)
• Harnessing the Computation Redundancy in ViTs to Boost Adversarial Transferability (NeurIPS 2025 · similarity 0.88)
• Attacking Transformers with Feature Diversity Adversarial Perturbation (AAAI 2024 · similarity 0.88)
• Vision Transformers Beat WideResNets on Small Scale Datasets Adversarial Robustness (AAAI 2025 · similarity 0.87)
• Improving Interpretation Faithfulness for Vision Transformers (ICML 2024 · similarity 0.87)