Research map / Multi-agent systems and games / LLM agents
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that a single adversarial image injected into one MLLM agent's memory can spread through randomized pairwise chats and jailbreak nearly all agents of a simulated multi-agent system (up to one million LLaVA-1.5 agents), exponentially fast.
Problem
Multimodal LLM agents with memory banks and multi-agent chat can be exploited; the paper asks whether a jailbreak of one agent can spread to the rest of a large agent population without further adversary intervention.
Approach
Introduces "infectious jailbreak": an adversarial image that is retrieved from an agent's image album during chat is stored in the answering agent's album, so the jailbreak self-propagates. The paper models the spread with an SIS-like infection dynamic, derives a logarithmic chat-round scaling, and states a simple condition (beta <= 2gamma) for provable restraint.
Key results
- Injecting one adversarial image into a single agent's album drives cumulative infection of 93.75% at the 16th chat round in a 256-agent low-diversity setting with border attack h=6.
- Simulations with up to one million LLaVA-1.5 agents (run with 1024 initial virus carriers) reach roughly 95-96% current infection by the 22nd round.
- Visual and textual prompt injection baselines fail to infect any agent, and sequential jailbreak only reaches about 1/8 of agents after 32 rounds.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adding the infectious adversarial image (vs. VP/TP/sequential noninfectious baselines) | multi-agent system with LLaVA-1.5 7B agents · N=256 simulation | cumulative infection ratio: infectious reaches near-complete infection exponentially; baselines do not | Helped |
| Visual prompt injection and textual prompt injection baselines | multi-agent jailbreak · N=256 simulation | cumulative infection ratio | Hurt |
| Increase text history memory |H| from 3 to 9 or 15 (ablation) | infectious jailbreak · Border h=6, N=256 | cumulative p16: 85.62 to 93.12 (|H|=3 to 9) | No clear effect |
| Reduce image album size |B| from 10 to 2 (ablation) | infectious jailbreak · Border h=6, N=256 | cumulative p16: 85.62 to 76.17; time to 90% rises from 16.60 to 19.40 | Hurt |
| Larger perturbation budget (border h=8 vs h=6, or pixel eps 16/255 vs 8/255) (ablation) | infectious jailbreak · High diversity, N=256 | cumulative p16: border h=6 88.98 vs h=8 91.95 (high diversity) | Helped |
| High textual chat diversity instead of low diversity | infectious jailbreak · Border h=6, N=256 | cumulative p16 and rounds to 90%: low 93.75 vs high 88.98 cumulative p16 | Hurt |
| Add momentum to BIM when crafting adversarial image (ablation) | adversarial image optimization · Best-epoch cumulative p16, N=256 | cumulative p16 (%): 58.59 to 87.89 | Helped |
| Add momentum to PGD when crafting adversarial image (ablation) | adversarial image optimization · Best-epoch cumulative p16, N=256 | cumulative p16 (%): 84.77 to 89.45 | Helped |
| Random image corruption augmentations (resize, flip, JPEG) during crafting (ablation) | adversarial image optimization · Corruption-robust setting, N=256 | infection ratio | No clear effect |
| Scale multi-agent system to one million agents (N=2^20) | infectious jailbreak · LLaVA-1.5 7B, c0=1/1024 | current infection ratio at round 22: p22=96.23% | Helped |
| Use InstructBLIP 7B backbone instead of LLaVA-1.5 | infectious jailbreak · InstructBLIP 7B agents, N=256 | infection ratio | No clear effect |
Surprising findings
- Visual and textual prompt injection baselines cannot jailbreak even a single agent in the simulation.
- Adversarial images crafted on the first 512 chat records generalize, with infection rates robust to larger text-history memory sizes.
Limitations
- Simulation uses a proof-of-concept randomized pairwise chat with exact-match harmful criteria, which may underestimate or misrepresent real-world effect.
- Infection dynamics derive from ideal mean-field assumptions with amortized alpha, beta, gamma constants.
- No practical defense is proposed; the provable-defense condition is only a principle.
Experimental setup
Datasets: AdvBench ArtBench acnh-images image pool names-dataset
Benchmarks: AdvBench harmful strings (manual evaluation of alignment)
Metrics: cumulative infection ratio current infection ratio first chat round reaching 90% infection JSR minCLIP
Baselines: Visual prompt injection Textual prompt injection Sequential jailbreak
Builds on: MI-FGSM CLIP dense retrieval AdvBench Visual prompt injection
Closest papers by meaning
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.88)
- AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents (ICLR 2025 · similarity 0.87)
- Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation (ICLR 2024 · similarity 0.87)
- SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism (NeurIPS 2025 · similarity 0.87)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.87)
- Adversarial Reasoning at Jailbreaking Time (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.