AI Paper Atlas

Research map / Adversarial robustness and security / Attacks on text and graphs

Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models

Dong Lu, Zhiqiang Wang, Teng Wang, Weili Guan, Hongchang Gao, Feng Zheng

ICCV 2023 · Adversarial robustness and security › Attacks on text and graphs · approach: Vision-language model robustness · 103 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace single image-text pair with set-level augmentation (multi-scale images, multiple captions) and cross-modal guidancemultimodal adversarial attack on ALBEF · Flickr30K ALBEF to TCLTR R@1 attack success rate: 15.21 -> 45.42Improved
same set-level guidance attack in white-box settingALBEF · Flickr30KTR R@1 attack success rate: 77.16 -> 97.24Improved
add MI transfer attack to Co-Attack (ablation)Co-Attack · Flickr30K ALBEF white-boxTR R@1 attack success rate: 77.16 -> 64.86Hurt
add unimodal transfer attacks (MI, DIM, PNA_PO) to Sep-Attack (ablation)Sep-Attack · Flickr30K, ALBEF sourceattack success rate: TR R@1 white-box 65.69 -> 58.81 (MI)Hurt
add DIM to Co-Attack (ablation)Co-Attack · Flickr30K ALBEF to TCLTR R@1 attack success rate: 15.21 -> 22.23 black-box; 77.16 -> 47.03 white-boxMixed
use SGA on cross-task transfer from retrieval to image captioningBLIP captioning · MSCOCOCIDEr: 125.5 -> 116.0 (lower is stronger attack)Improved
use SGA on cross-task transfer to visual groundingALBEF visual grounding · RefCOCO+ Valaccuracy: 54.26 -> 53.55 (lower is stronger attack)Improved
increase number of multi-scale image copies (ablation)SGA · Flickr30K ALBEF to CLIP_ViTattack success rateImproved
increase caption set size beyond one (ablation)SGA · Flickr30K ALBEF to CLIP_ViTattack success rateImproved
SGA versus Sep-Attack on CLIP_ViT image retrieval black-boxSGA · Flickr30K ALBEF to CLIP_ViTIR R@1 attack success rate: 45.23 -> 44.16Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy