Research map / Image generation / Text-to-image diffusion
UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UPAM, a gradient-trained LLM+LoRA attack model that rewrites naive prompts into natural adversarial prompts that bypass both textual filters and CLIP-based visual checkers of black-box T2I APIs.
Problem
Black-box text-to-image APIs deploy both textual filters and visual checkers that block harmful prompts and images, and existing enumeration-based attacks are inefficient, mostly target only the textual filter, and produce unnatural prompts.
Approach
Trains a parameterized LLM-based attack model (frozen LLaMA + LoRA) with zeroth-order gradient estimates. Sphere-Probing Learning estimates gradients from binary Pass/Deny outputs when no image is returned, and Semantic-Enhancing Learning refines returned images toward target semantics via CLIP losses, with gradient harmonization to keep the model in the Pass region.
Key results
- Under Protocol A, UPAM reaches 38.37% R-1 precision on DALL·E versus 8.65% for the best baseline (RIATIG), with inference time of about 5 s per sample.
- Under Protocol B, UPAM achieves 7.3/10 R-1 on DALL·E while baselines such as TextFooler and HomoSubs get 0/10.
- Ablation: removing SPL drops R-1 on DALL·E to 0.22%, and removing SEL drops it to 23.25%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UPAM instead of RIATIG | DALL·E · Protocol A | R-1 Precision: 8.65% -> 38.37% | Helped |
| Use UPAM instead of RIATIG | DALL·E 2 · Protocol A | R-1 Precision: 8.55% -> 40.45% | Helped |
| Use UPAM instead of RIATIG | Imagen · Protocol A | R-1 Precision: 7.80% -> 36.16% | Helped |
| Add SPL to UPAM (ablation) | UPAM · DALL·E, Protocol A | R-1: 0.22% -> 38.37% | Helped |
| Add SEL to UPAM (ablation) | UPAM · DALL·E, Protocol A | R-1: 23.25% -> 38.37% | Helped |
| Add Moving Closer to Boundary to SPL (ablation) | UPAM · DALL·E, Protocol A | R-1: 35.76% -> 38.37% | Helped |
| Add Gradient Harmonization to SEL (ablation) | UPAM · DALL·E, Protocol A | R-1: 33.82% -> 38.37% | Helped |
| Use a pretrained LLM instead of an untrained transformer (ablation) | UPAM · DALL·E, Protocol A | PPL: 3954.85 -> 706.32 | Helped |
| Optimize LoRA rather than the pretrained LLM directly (ablation) | UPAM · DALL·E, Protocol A | PPL: 2848.52 -> 706.32 | Helped |
Surprising findings
- Enumeration baselines such as TextFooler and HomoSubs rarely get the black-box API to return any image, so their R-precision is near zero.
Limitations
- Visual checker is a CLIP-embedding threshold filter with a blacklist textual filter, a simplified stand-in for real deployed defenses.
- Harmful targets are COCO classes rather than truly harmful content, for ethical reasons.
Experimental setup
Datasets: MS COCO
Benchmarks: MS COCO (10 classes)
Metrics: R-1 precision R-3 precision textual similarity inference time perplexity (GPT-2)
Baselines: TextFooler HomoSubs EvoPrompt HiddenVocab MACPrompt RIATIG
Builds on: ZOO (zeroth-order optimization) LoRA CLIP RIATIG
Closest papers by meaning
- Ask, Attend, Attack: An Effective Decision-Based Black-Box Targeted Attack for Image-to-Text Models (NeurIPS 2024 · similarity 0.86)
- Distraction is All You Need: Memory-Efficient Image Immunization against Diffusion-Based Image Editing (CVPR 2024 · similarity 0.86)
- Image Hijacks: Adversarial Images can Control Generative Models at Runtime (ICML 2024 · similarity 0.85)
- OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models (NeurIPS 2025 · similarity 0.85)
- When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models (ICML 2026 · similarity 0.85)
- MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMs (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.