Research map / Adversarial robustness and security / Attacks on text and graphs
VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VLAttack rather than the strongest listed baseline | ViLT · VQAv2 and NLVR2 | ASR (%): 65.20 -> 78.05; 52.17 -> 66.65 | Helped |
| Use VLAttack rather than the strongest listed baseline | BLIP · VQAv2 and NLVR2 | ASR (%): 25.04 -> 48.78; 33.08 -> 52.66 | Helped |
| Use VLAttack rather than the strongest listed baseline | Unitab · VQAv2, RefCOCO, RefCOCO+, and RefCOCOg | ASR (%): 48.40 -> 62.20; 89.70 -> 93.52; 90.96 -> 93.40; 91.31 -> 95.61 | Helped |
| Use VLAttack rather than the strongest listed baseline | OFA · VQAv2, SNLI-VE, RefCOCO, RefCOCO+, and RefCOCOg | ASR (%): 51.16 -> 78.82; 29.19 -> 41.78; 32.04 -> 56.62; 45.28 -> 58.14; 54.63 -> 73.30 | Helped |
| Use BSA rather than the strongest listed image-attack baseline | OFA · MSCOCO | CIDEr: 23.54 -> 2.16 | Helped |
| Use BSA rather than SSP | CLIP-RN50 · SVHN | ASR (%): 84.26 -> 84.98 | Helped |
| Add Transformer-encoder representations to image-encoder features in the BSA loss (ablation) | BSA · Multimodal tasks and image-only tasks | ASR (or captioning score) | Helped |
| Add BERT-Attack text perturbations after BSA (ablation) | BSA · Multimodal tasks | ASR | Helped |
| Add iterative cross-search instead of simply querying each image-text candidate pair (ablation) | BSA+BA · Multimodal tasks | ASR | Helped |
| Increase the total iteration budget from 10 toward 40 (ablation) | VLAttack on ViLT · VQAv2 | ASR | Helped |
| Increase the initial image-attack iterations from 5 toward 40 (ablation) | VLAttack on ViLT · VQAv2 | ASR | Hurt |
| Replace projected-gradient optimization with Momentum Iterative optimization (ablation) | VLAttack · ViLT and Unitab tasks | ASR: +9.70% average ASR improvement over BSA_MI | Helped |
Closest papers by meaning
- HQA-VLAttack: Towards High Quality Adversarial Attack on Vision-Language Pre-Trained Models (NeurIPS 2025 · similarity 0.93)
- VQAttack: Transferable Adversarial Attacks on Visual Question Answering via Pre-trained Models (AAAI 2024 · similarity 0.91)
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.90)
- Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.89)
- Anyattack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models (CVPR 2025 · similarity 0.89)
- Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.