Research map / Adversarial robustness and security / Attacks on text and graphs
Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models
Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, Feng Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single image-text pair with set-level augmentation (multi-scale images, multiple captions) and cross-modal guidance | multimodal adversarial attack on ALBEF · Flickr30K ALBEF to TCL | TR R@1 attack success rate: 15.21 -> 45.42 | Improved |
| same set-level guidance attack in white-box setting | ALBEF · Flickr30K | TR R@1 attack success rate: 77.16 -> 97.24 | Improved |
| add MI transfer attack to Co-Attack (ablation) | Co-Attack · Flickr30K ALBEF white-box | TR R@1 attack success rate: 77.16 -> 64.86 | Hurt |
| add unimodal transfer attacks (MI, DIM, PNA_PO) to Sep-Attack (ablation) | Sep-Attack · Flickr30K, ALBEF source | attack success rate: TR R@1 white-box 65.69 -> 58.81 (MI) | Hurt |
| add DIM to Co-Attack (ablation) | Co-Attack · Flickr30K ALBEF to TCL | TR R@1 attack success rate: 15.21 -> 22.23 black-box; 77.16 -> 47.03 white-box | Mixed |
| use SGA on cross-task transfer from retrieval to image captioning | BLIP captioning · MSCOCO | CIDEr: 125.5 -> 116.0 (lower is stronger attack) | Improved |
| use SGA on cross-task transfer to visual grounding | ALBEF visual grounding · RefCOCO+ Val | accuracy: 54.26 -> 53.55 (lower is stronger attack) | Improved |
| increase number of multi-scale image copies (ablation) | SGA · Flickr30K ALBEF to CLIP_ViT | attack success rate | Improved |
| increase caption set size beyond one (ablation) | SGA · Flickr30K ALBEF to CLIP_ViT | attack success rate | Improved |
| SGA versus Sep-Attack on CLIP_ViT image retrieval black-box | SGA · Flickr30K ALBEF to CLIP_ViT | IR R@1 attack success rate: 45.23 -> 44.16 | Hurt |
Closest papers by meaning
- Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory (ECCV 2024 · similarity 0.93)
- Transform to Transfer: Boosting Adversarial Attack Transferability on Vision-Language Pre-training Models (CVPR 2026 · similarity 0.93)
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.90)
- Failures to Find Transferable Image Jailbreaks Between Vision-Language Models (ICLR 2025 · similarity 0.89)
- Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.89)
- Mutual-Modality Adversarial Attack with Semantic Perturbation (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
