Research map / Adversarial robustness and security / Backdoor and poisoning

Attention-Imperceptible Backdoor Attacks on Vision Transformers

AAAI 2025 · Adversarial robustness and security › Backdoor and poisoning · approach: Backdoors in large multimodal models · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AIBA, a ViT backdoor attack that places an L-infinity-bounded adversarial patch trigger at the most-attended patches and alternates trigger and model updates per mini-batch, achieving ~99.97% ASR with low attention-level anomalies.

Problem

Existing ViT-specific backdoor attacks use conspicuous triggers and optimize triggers to maximize attention, creating attention-map anomalies that make them detectable by ViT-specific defenses.

Approach

Attach a norm-constrained adversarial perturbation trigger to the top-attention patches (via attention rollout) and optimize only the trigger-to-target mapping with a mini-batch-level alternating bi-level scheme, so the trigger does not distort the model's attention.

Key results

• On DeiT-Base with ImageNet, AIBA reaches 99.97% ASR and 81.54% BA (CA 81.80), versus 96.03% to 99.76% for baselines.
• AIBA attention-level stealthiness: ARES 0.078, APSNR 20.68, ALPIPS 0.000058 on DeiT-Base/ImageNet, with much lower ALPIPS than baselines.
• Against the BDVT defense, ASR stays at 99.80% when masking top-4 patches; PatchProcess TPR stays below 4%.

Measured effects reported in the paper

Surprising findings

• WaNet achieves lower ARES (0.011) than AIBA (0.078), though the authors attribute this to not interfering with attention during training.
• Trigger perturbation budget below 2/255 still yields 99.82% ASR.

Limitations

• PatchShuffle threshold in PatchProcess evaluation was 0, so that defense was only partially analyzed.
• Trade-off: more trigger patches raise ASR but increase attention anomalies (ARES) and reduce APSNR.
• BA drops more for smaller models (0.42% for DeiT-Tiny) on ImageNet.

Experimental setup

Datasets: ImageNet CIFAR-10

Benchmarks: ImageNet CIFAR-10

Metrics: clean accuracy backdoor accuracy attack success rate attention residual (ARES) attention PSNR attention LPIPS TPR TNR

Baselines: BadNet WaNet DBIA TrojViT BadViT BDVT PatchProcess

Builds on: PGD adversarial training Attention Rollout BadNets DBIA TrojViT

Closest papers by meaning

• Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models (CVPR 2025 · similarity 0.89)
• Not All Prompts Are Secure: A Switchable Backdoor Attack Against Pre-trained Vision Transfomers (CVPR 2024 · similarity 0.88)
• Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift (CVPR 2025 · similarity 0.88)
• Towards Understanding and Improving Adversarial Robustness of Vision Transformers (CVPR 2024 · similarity 0.88)
• BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization (NeurIPS 2025 · similarity 0.88)
• A Dual Stealthy Backdoor: From Both Spatial and Frequency Perspectives (AAAI 2024 · similarity 0.87)