Research map / Adversarial robustness and security / Black-box and transfer attacks
AGS: Affordable and Generalizable Substitute Training for Transferable Adversarial Attack
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a ResNet-50 substitute model from scratch on unlabeled out-of-domain images using adversary-centric contrastive learning plus adversarial invariant learning, then crafts transferable attacks from its intermediate features.
Problem
Transfer-based black-box attacks typically use ImageNet-pretrained supervised substitute models, which are expensive to obtain and overfit to the classification task; the paper seeks affordable, generalizable substitutes trained without labels.
Approach
Adds a slightly adversarial view as an extra positive in contrastive learning (adversary-centric contrastive learning) so benign and adversarial views interact, and adds an adversarial invariant learning regularizer that makes adversarial representations invariant across augmentation strengths, yielding a flatter loss landscape and better transferability.
Key results
- On COCO-trained substitute, AGS reaches 1.54 vs 2.23 average CNN accuracy and about 16.71 average ViT accuracy vs 27.75 for pretrained ResNet50 (lower is better), outperforming CL+adv (21.64 on ViTs).
- AGS achieves 6.43% average accuracy on CNNs and 25.23% on ViTs against robust-model-related settings was not reported; on robust CNNs (Tab. 4) AGS (COCO) gets 12.74 on Inc-v3ens3 vs 55.36 for ResNet50.
- On DETR object detection, AGS obtains mAP 2.8 vs 2.6 for pretrained ResNet50 and beats best compared No-box/APR by an average margin of about 9.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adversary-centric contrastive learning with adversarial and benign views instead of classical contrastive learning with adversarial views (ablation) | AGS substitute training · ImageNet validation, COCO-trained substitute, CNNs (Avg.) | Top-1 accuracy (%): 6.13 -> 2.75 | Helped |
| Use adversary-centric contrastive learning with adversarial and benign views instead of classical contrastive learning with adversarial views (ablation) | AGS substitute training · ImageNet validation, COCO-trained substitute, ViTs (Avg.) | Top-1 accuracy (%): 21.64 -> 19.78 | Helped |
| Add adversarial invariant learning (AIL) (ablation) | ACCL substitute training · ImageNet validation, COCO-trained substitute, CNNs (Avg.) | Top-1 accuracy (%): 2.75 -> 2.23 | Helped |
| Add adversarial invariant learning (AIL) (ablation) | ACCL substitute training · ImageNet validation, COCO-trained substitute, ViTs (Avg.) | Top-1 accuracy (%): 19.78 -> 16.71 | Helped |
| Use adversarial views in ACCL rather than replacing them with original examples (ablation) | AGS substitute training · ImageNet validation, COCO-trained substitute, CNNs (Avg.) | Top-1 accuracy (%): 5.34 -> 2.75 | Helped |
| Use adversarial views in ACCL rather than replacing them with original examples (ablation) | AGS substitute training · ImageNet validation, COCO-trained substitute, ViTs (Avg.) | Top-1 accuracy (%): 31.15 -> 19.78 | Helped |
| Train the substitute with AGS rather than a classical contrastive-learning baseline with adversarial training | ResNet-50 substitute · ImageNet validation, COCO-trained substitute, CNNs (Avg.) | Top-1 accuracy (%): 6.13 -> 2.23 | Helped |
| Train the substitute with AGS rather than a classical contrastive-learning baseline with adversarial training | ResNet-50 substitute · ImageNet validation, COCO-trained substitute, ViTs (Avg.) | Top-1 accuracy (%): 21.64 -> 16.71 | Helped |
| Train the substitute with AGS rather than ImageNet-pretrained ResNet-50 | ResNet-50 substitute · ImageNet validation, COCO-trained substitute, ViTs (Avg.) | Top-1 accuracy (%): 27.75 -> 16.71 | Helped |
| Train the substitute with AGS rather than ImageNet-pretrained ResNet-50 | ResNet-50 substitute · ImageNet validation, COCO-trained substitute, CNNs (Avg.) | Top-1 accuracy (%): 1.54 -> 2.23 | Hurt |
| Train the substitute with AGS on COCO rather than ImageNet-pretrained ResNet-50 | ResNet-50 substitute · Inc-v3ens3 | Top-1 accuracy (%): 55.36 -> 12.74 | Helped |
| Train the substitute with AGS on COCO rather than ImageNet-pretrained ResNet-50 | ResNet-50 substitute · Inc-v3ens4 | Top-1 accuracy (%): 57.48 -> 17.64 | Helped |
Surprising findings
- A substitute trained from scratch on only unlabeled out-of-domain images (e.g., COCO 40k) outperforms an ImageNet-pretrained ResNet50 against ViTs.
- Using only 10% of COCO (40k) already beats the best competitor (APR R) on CNNs by over 5%.
Limitations
- Gains over the pretrained ResNet50 on CNN targets are small; the pretrained ResNet50 is slightly better on DETR mAP in some settings.
- Performance on the Comics substitute degrades on video segmentation (J&F recall 31.5 vs 47.9 for ResNet50), showing sensitivity to training data domain.
Experimental setup
Datasets: COCO (40k unlabeled) Comics (50k unlabeled) Paintings (79k unlabeled) ImageNet validation set (5k evaluation images)
Benchmarks: ImageNet validation (5k adversarial examples) MS COCO validation (DETR) DAVIS validation (DINO)
Metrics: Top-1 accuracy mAP mAR J&F mean J&F recall
Baselines: No-box (R) No-box (J) APR (R) APR (J) ResNet50 (ImageNet-pretrained) CL CL+adv
Builds on: SimCLR No-box attack APR (adversarial pixel restoration) MI-FGSM invariant learning
Closest papers by meaning
- DaST: Data-Free Substitute Training for Adversarial Attacks (CVPR 2020 · similarity 0.88)
- Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examples (ICLR 2023 · similarity 0.88)
- Enhancing Adversarial Transferability with Adversarial Weight Tuning (AAAI 2025 · similarity 0.85)
- Boosting Adversarial Transferability through Augmentation in Hypothesis Space (CVPR 2025 · similarity 0.85)
- PubDef: Defending Against Transfer Attacks From Public Models (ICLR 2024 · similarity 0.85)
- Improving Transferable Targeted Adversarial Attacks with Model Self-Enhancement (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.