Research map / Adversarial robustness and security / Attacks on text and graphs
Dissecting Adversarial Robustness of Multimodal LM Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds VWA-Adv, 200 targeted adversarial web tasks on VisualWebArena, and an agent-graph framework (ARE) that decomposes attack success into edge weights to show how multimodal web agents built on black-box VLMs can be hijacked via single-image perturbations.
Problem
Adversarial robustness of compound multimodal LM web agents (with captioners, evaluators, value functions, reflexion, tree search) under realistic targeted attacks injected via a single user-controlled element of the environment.
Approach
Model an agent as a directed graph of components and quantify adversarial influence along edges as tightest upper bound on attack success, so robustness of each component and of added components (evaluators, value functions) can be decomposed and reused across configurations.
Key results
- Black-box CLIP attack with a single image perturbation (epsilon 16/256) achieves up to 67% attack success rate against GPT-4V-based agents.
- GPT-4o-based policy model reaches 31% ASR under captioner white-box attack and 40% ASR under text prompt injection.
- Attacking the reflexion evaluator together with the policy model raises ASR from 31% (base agent) to 36%, while attacking the tree search value function raises ASR from 31% to 38%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Attack the white-box captioner instead of using no captioner | GPT-4o policy agent · VWA-Adv, image access | ASR | Mixed |
| Add an uncompromised evaluator and reflection | Base agent · VWA-Adv, reflexion | ASR: 31% to 25% | Helped |
| Attack both the evaluator and policy model | Reflexion agent · VWA-Adv | ASR: 31% to 36% | Hurt |
| Attack the evaluator while leaving the policy model uncompromised | Reflexion agent · VWA-Adv | ASR: 8% | Hurt |
| Add an uncompromised value function | Base agent · VWA-Adv, tree search | ASR: 31% to 26% | Helped |
| Attack both the value function and policy model | Tree search agent · VWA-Adv | ASR: 31% to 38% | Hurt |
| Attack the value function while leaving the policy model uncompromised | Tree search agent · VWA-Adv | ASR: 8% | Hurt |
| Add a system prompt prioritizing visual inputs | GPT-4o captioner-augmented agent · VWA-Adv, captioner attack | ASR | No clear effect |
| Prompt the model to stop when it detects inconsistencies or adversarial instructions | GPT-4o captioner-augmented agent · VWA-Adv, captioner attack | ASR | No clear effect |
| Paraphrase untrusted text with GPT-4o | GPT-4o captioner-augmented agent · VWA-Adv, captioner attack | ASR: 31% to 27.5% | Helped |
| Caption screenshot images separately and override inconsistent text | GPT-4o captioner-augmented agent · VWA-Adv, captioner attack | ASR: near-zero | Helped |
| Remove negative text from the CLIP attack objective (ablation) | CLIP attack · GPT-4V targeted caption generation | Targeted cap.: 71% to 46% | Hurt |
Surprising findings
- Inference-time compute methods that improve benign performance (reflexion, tree search) can reduce robustness when their evaluator or value function is compromised.
- Uncompromised evaluators and value functions can improve robustness by blocking adversarial actions, creating a false sense of security.
Limitations
- Attack baselines are engineered versions of existing attacks and may be lower bounds on risk.
- Evaluation limited to the VisualWebArena web environment; other settings such as operating systems untested.
- Only a few agent architectures (base, reflexion, tree search) are considered.
Experimental setup
Datasets: VisualWebArena
Benchmarks: VWA-Adv VisualWebArena
Metrics: attack success rate benign success rate edge weight
Baselines: base agent captioner-augmented agent reflexion agent tree search agent GPT-4V agent Gemini-1.5-Pro agent Claude-3-Opus agent
Builds on: VisualWebArena reflexion tree search PGD instruction hierarchy
Closest papers by meaning
- On the Robustness of Large Multimodal Models Against Image Adversarial Attacks (CVPR 2024 · similarity 0.88)
- Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents (ICLR 2026 · similarity 0.87)
- When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems (ICML 2026 · similarity 0.87)
- AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition (ICML 2026 · similarity 0.87)
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (NeurIPS 2024 · similarity 0.86)
- On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.