Research map / Adversarial robustness and security / Attacks on text and graphs

Image Hijacks: Adversarial Images can Control Generative Models at Runtime

ICML 2024 · Adversarial robustness and security › Attacks on text and graphs · approach: Vision-language model robustness · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces image hijacks, adversarial images that control VLM behaviour at inference time, via Behaviour Matching and Prompt Matching, evaluated on LLaVA-13B across specific string, leak-context, jailbreak and disinformation attacks.

Problem

Vision-language models accept continuous image inputs, creating a new channel for adversarial control; it is unknown whether image inputs can hijack VLM behaviour (output, leakage, safety bypass, false beliefs) at inference time.

Approach

Behaviour Matching optimises an adversarial image so the VLM's per-token logits match a target behaviour across a large context set; Prompt Matching extends this by using the logits produced under a target text prompt as soft targets, so images can mimic arbitrary text prompts without needing to match embeddings.

Key results

• Specific string hijacks reach 100% success with unconstrained or epsilon=32/255 image perturbations, and 95% with a 60x60 stationary patch.
• Leak-context attack achieves up to 96% success (epsilon=32/255), while the GCG text baseline gets 0% on leak context.
• Disinformation Prompt Matching unconstrained images reach 85% success versus 0% for the untrained baseline; jointly trained LLaVA+InstructBLIP hijack gets 0% transfer to BLIP-2.

Measured effects reported in the paper

Surprising findings

• Image hijacks trained on one white-box VLM show 0% success when transferred to a held-out black-box model (BLIP-2), even though ensembled training yields high success on the training models.
• The GCG text baseline achieves 82% jailbreak success, higher than image jailbreaks under some constraints, while image attacks outperform it on specific string and leak context.

Limitations

• Attacks assume white-box access to open-source VLMs; transfer to black-box models failed in the reported experiments.
• Jailbreak images overfit late in training, producing affirmative responses without substantive help.
• Disinformation prompt-matching falls short of the target prompt's performance (85% vs 100%) and did not transfer to held-out models.

Experimental setup

Datasets: Alpaca AdvBench ImageNet

Benchmarks: Alpaca held-out instructions (specific string, leak context) AdvBench-based jailbreak test set disinformation validation/test questions

Metrics: attack success rate context transfer rate Levenshtein edit distance validation loss

Baselines: Greedy Coordinate Gradient (GCG) original unmodified image untrained image

Builds on: Greedy Coordinate Gradient knowledge distillation Alpaca instruction tuning

Closest papers by meaning

• Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.90)
• Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models (CVPR 2025 · similarity 0.89)
• Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.88)
• Backdooring Vision-Language Models with Out-Of-Distribution Data (ICLR 2025 · similarity 0.87)
• FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts (AAAI 2025 · similarity 0.86)
• A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1 (NeurIPS 2025 · similarity 0.86)