Research map / Adversarial robustness and security / Attacks on text and graphs
Attention! Your Vision Language Model Could Be Maliciously Manipulated
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VMA, an image-based adversarial attack that uses Adam-style momentum and a sigmoid reparameterization to precisely control the full output token sequence of VLMs, and demonstrates it on jailbreaking, hijacking, hallucination, privacy, denial-of-service, sponge, and watermarking tasks.
Problem
VLMs are vulnerable to adversarial inputs, but prior attacks mainly elicit harmful or wrong content rather than precisely controlling every output token; the paper asks whether imperceptible image perturbations can force an exact target output sequence.
Approach
Argues visual inputs are continuous and thus more sensitive than discrete text, then optimizes an image perturbation with Adam-style first- and second-order momentum over a sigmoid-based differentiable reparameterization of the epsilon-ball, replacing PGD's sign-gradient projection.
Key results
- Manipulation achieves an average ASR of 90.25% across VLMs, images, and prompts.
- Jailbreaking reaches at least 88.05% ASR on AdvBench and 80.84% on MM-SafetyBench at epsilon=4/255.
- Denial-of-service achieves at least 88.20% ASR at epsilon=4/255, and sponge examples increase inference time over 200x on DeepSeek-VL and 400x on other models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase perturbation budget from ε=0/255 to ε=4/255 | VMA on hallucination tasks · Hallucination | ASR (%): 0.34 -> 100.00 | Helped |
| Increase perturbation budget from ε=0/255 to ε=4/255 | VMA on privacy-breach tasks · Privacy breach | ASR (%): 0.00 -> 99.09 | Helped |
| Increase perturbation budget from ε=0/255 to ε=4/255 | VMA on denial-of-service tasks · Denial-of-Service | ASR (%): 0.62 -> 93.79 | Helped |
| Increase perturbation budget from ε=0/255 to ε=4/255 | VMA on hijacking tasks · VQA-CLS | ASR (%): 33.33 -> 76.28 | Helped |
| Increase perturbation budget from ε=0/255 to ε=16/255 | VMA watermarking · Watermarking | Protection rate (%): 0.00 -> 99.18 | Helped |
| Set learning rate to 0.1 rather than 0.001 (ablation) | VMA on Phi3 · Manipulation | ASR (%): 10.00 -> 90.00 | Helped |
| Set learning rate to 0.3 rather than 0.1 (ablation) | VMA on Phi3 · Manipulation | ASR (%): 90.00 -> 91.00 | Helped |
| Set momentum coefficient to 0.9 rather than 0.5 (ablation) | VMA on Phi3 · Manipulation | ASR (%): 87.0 -> 90.0 | Helped |
| Use differentiable transformation (ablation) | VMA without momentum · Manipulation | ASR (%): 71.0 -> 79.0 | Helped |
| Use momentum (ablation) | VMA without differentiable transformation · Manipulation | ASR (%): 71.0 -> 88.0 | Helped |
| Add differentiable transformation and momentum together (ablation) | VMA on Phi3 · Manipulation | ASR (%): 71.0 -> 96.0 | Helped |
| Add differentiable transformation and momentum together (ablation) | VMA on Phi3 · Manipulation | Average number of iterations: 104 -> 26 | Helped |
Surprising findings
- Clean-image hijacking already reaches 81.37% (Phi3) and 67.56% (DeepSeek-VL) ASR on CLS-CPT at epsilon=0 due to task ambiguity.
- LLaVA is highly jailbreakable without any perturbation (over 40% ASR at epsilon=0) while the other three models are robust.
Limitations
- Poor transferability across different model architectures, limiting real-world black-box use.
- Difficulty increases as the target output sequence length grows.
- Requires white-box access to gradients and output probabilities.
Experimental setup
Datasets: AdvBench MM-SafetyBench COCO POPE MLLMU-Bench MM-Vet Van Gogh paintings
Benchmarks: AdvBench MM-SafetyBench COCO POPE MLLMU-Bench MM-Vet
Metrics: attack success rate protection rate average inference time output token count
Baselines: PGD
Builds on: projected gradient descent Adam Image Hijacks staircase mechanism randomized smoothing
Closest papers by meaning
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.91)
- Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models (CVPR 2025 · similarity 0.91)
- Pandora's Box: Towards Building Universal Attackers against Real-World Large Vision-Language Models (NeurIPS 2024 · similarity 0.91)
- Image Hijacks: Adversarial Images can Control Generative Models at Runtime (ICML 2024 · similarity 0.90)
- Visual Adversarial Examples Jailbreak Aligned Large Language Models (AAAI 2024 · similarity 0.90)
- Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.