Research map / Adversarial robustness and security / Attacks on text and graphs
Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that image-space adversarial examples uploaded to the web can hijack benign users of IP-Adapter-equipped text-to-image diffusion models into triggering NSFW outputs, and explores adversarially trained CLIP as a defense.
Problem
Image Prompt Adapters in text-to-image diffusion models open a new jailbreak vector: benign users may unknowingly download and query with adversarial images, causing NSFW outputs and reputational harm to service providers.
Approach
Attack Encoder Only (AEO) crafts imperceptible adversarial images by aligning their features in the image encoder (CLIP or face recognition model) with an NSFW image prompt, exploiting the IP-Adapter's dependence on frozen open-source encoders, and proposes adversarially trained CLIP (FARE) to defend.
Key results
- AEO promotes Nudity rate to at least 57.3% and NSFW rate to at least 77% across T2I-IP-DMs at weight factor 1.0 on text-to-image.
- On virtual try-on (IDM-VTON), AEO achieves 56.2% Nudity rate and 83.3% NSFW rate.
- Concept erasing methods (ESD-u, NP, SLD) fail to raise Adversarial CLIP Score, while FARE-finetuned CLIP promotes it and preserves fidelity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AEO (cosine similarity) rather than a benign image prompt | T2I-IP-DMs · Text-to-image, weight factor 1.00 | NSFW rate (%): 4.50 -> 95.30 | Hurt |
| increase the weight factor from 0.25 to 1.00 for AEO (COS) (ablation) | T2I-IP-DMs · Text-to-image | NSFW rate (%): 21.70 -> 95.30; 1.10 -> 77.00 | Mixed |
| use cosine similarity instead of MSE for AEO (ablation) | T2I-IP-DMs · Text-to-image, weight factor 1.00 | NSFW rate (%): 95.30 vs 95.30; 67.60 vs 77.00 | Mixed |
| use AEO (COS) rather than a benign image prompt | T2I-IP-DMs · Image inpainting | ISM: 0.05 -> 0.61 | Helped |
| use AEO (COS) rather than a benign image prompt | IDM-VTON · Virtual try-on | NSFW rate (%): 5.40 -> 83.30 | Hurt |
| use InsightFace rather than CLIP as the AEO surrogate (ablation) | SD-v1-5-PlusID · Image inpainting | ISM: 0.06 -> 0.37 | Helped |
| use InsightFace rather than CLIP as the AEO surrogate (ablation) | SDXL-PlusID · Image inpainting | ISM: 0.15 -> 0.57 | Helped |
| use CLIP rather than InsightFace as the AEO surrogate (ablation) | Kolors-PlusID · Image inpainting | ISM: 0.09 -> 0.13 | Helped |
| increase the weight factor for AEO (ablation) | T2I-IP-DMs · Text-to-image | NSFW rate (%) | Helped |
| replace the original CLIP image encoder with adversarially trained FARE (ablation) | SD-v1-5-Global · Text-to-image | Adversarial CLIP Score | Helped |
| apply SLD-Strong and SLD-Max concept erasing (ablation) | SD-v1-5-Global · Text-to-image | Benign CLIP Score | Hurt |
| apply ESD-u concept erasing (ablation) | SD-v1-5-Global · Text-to-image | NSFW rate (%) | Helped |
Surprising findings
- Adversarial images with imperceptible perturbations (epsilon 8/255) trigger NSFW outputs in IP-Adapter models even though they look like benign paintings.
- Existing filter and concept-erasing defenses cannot correct the unaligned behavior induced by adversarial examples without degrading fidelity.
Limitations
- Attack assumes adversary has access to open-source image encoders used by the IGS.
- Adversarial training may be bypassed under unseen threat models and FARE partly biases the encoder away from the original.
- Nudity and NSFW metrics rely on NudeNet and SD safety checker, which can misclassify images.
Experimental setup
Datasets: NudeNet SD safety checker
Benchmarks: Text-to-image paintings Image inpainting portraits Virtual try-on garments
Metrics: Nudity rate NSFW rate ISM CLIP Score Cosine similarity to concept text
Baselines: Benign image prompt Malicious NSFW image prompt Negative Prompt ESD-u SLD FARE
Builds on: IP-Adapter PGD adversarial attack FARE ESD SLD
Closest papers by meaning
- Image Hijacks: Adversarial Images can Control Generative Models at Runtime (ICML 2024 · similarity 0.85)
- T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attacks (NeurIPS 2025 · similarity 0.85)
- When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models (ICML 2026 · similarity 0.85)
- Perception-Guided Jailbreak Against Text-to-Image Models (AAAI 2025 · similarity 0.85)
- MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs (ICLR 2026 · similarity 0.84)
- Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.