Research map / Adversarial robustness and security / Certified and empirical robustness

Harnessing Edge Information for Improved Robustness in Vision Transformers

AAAI 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EdgeNet, a lightweight plug-and-play side branch that processes Canny edges of images and injects them via zero-convolution sandwich blocks into a frozen pretrained ViT-B/16 to improve adversarial robustness without hurting clean accuracy.

Problem

Improving adversarial robustness of high-accuracy pretrained vision transformers without sacrificing clean ImageNet accuracy, which prior robust fine-tuning (TORA-ViT) handles only via a accuracy-robustness trade-off hyperparameter.

Approach

Edge maps (from the Canny detector) carry relevant, robust shape/foreground features; a side branch of ViT blocks wrapped in zero convolutions injects these features into intermediate layers of a frozen backbone, with a joint clean+adversarial loss.

Key results

• EdgeNet-ViT-B/16 reaches 69.8% FGSM and 48.8% PGD accuracy vs 54.7% and 38.0% for TORA-ViT-B/16 (λ=0.5), with the same 83.7% clean accuracy.
• Replacing edge inputs with raw images lowers robustness (FGSM 64.4%, PGD 47.0%) versus edge input (69.8%, 48.8%).
• Under black-box transfer from ViT-B/16, EdgeNet raises accuracy from 35.03% to 74.41% (FGSM) and 14.26% to 70.32% (PGD).

Measured effects reported in the paper

Surprising findings

• Injecting raw image features into the side branch yields lower robustness than injecting Canny edges, attributed to redundancy with backbone features.

Limitations

• Robustness evaluated mainly with weak FGSM/PGD settings (epsilon 1/255, 5-step PGD) and FGSM-based adversarial training.
• Sensitive to trade-off hyperparameters alpha/beta and the insertion interval; 12-block configuration converges poorly.
• Clean accuracy on ImageNet variants slightly below TORA-ViT with lower lambda.

Experimental setup

Datasets: ImageNet-1K

Benchmarks: ImageNet-1K ImageNet-A ImageNet-R ImageNet-C

Metrics: top-1 accuracy FGSM accuracy PGD accuracy mCE

Baselines: TORA-ViT-B/16 RVT-B PyramidAT DeiT-B/16 Swin-B ConvNeXt-B ResNet-50 ViT-B/16 classification-head fine-tuning

Builds on: Adversarial training (FGSM) TORA-ViTs ControlNet zero convolution Adapter Canny edge detector

Closest papers by meaning

• Edges to Shapes to Concepts: Adversarial Augmentation for Robust Vision (CVPR 2023 · similarity 0.90)
• Towards Robust Vision Transformers: Path Dependency Analysis and a Simple Two-Stage Adversarial Training (CVPR 2026 · similarity 0.87)
• Does Robustness on ImageNet Transfer to Downstream Tasks? (CVPR 2022 · similarity 0.87)
• ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editing (CVPR 2023 · similarity 0.87)
• Explicitly Modeling Subcortical Vision with a Neuro-Inspired Front-End Improves CNN Robustness (NeurIPS 2025 · similarity 0.86)
• Vision Transformers Beat WideResNets on Small Scale Datasets Adversarial Robustness (AAAI 2025 · similarity 0.86)