Research map / Image generation / Text-to-image diffusion
Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAGE, an adversarial search method that uses robust classifier ensembles as surrogate losses to search the prompt, token-embedding, and latent spaces of text-guided diffusion models and discover four failure modes.
Problem
Text-guided diffusion models fail unexpectedly, both for natural-looking prompts producing wrong content and for latent samples or token embeddings producing distorted or unrelated images; prior failure discovery relied on manual prompts or CLIP-space gibberish and ignored the latent space.
Approach
Adversarial optimization over three input spaces: a robust ensemble of classifiers (including an edge-based classifier) as surrogate loss, a residual connection to an intermediate denoising step to get usable latent gradients, and a gradient-guided search that uses LLaMA word proposals to find natural-language prompts.
Key results
- On SD V2.1, SAGE reaches 80.7% human-verified search success rate in latent space versus 12.5% for PGD baseline, with NGR dropping from 75.0% to 12.7%.
- Token-embedding search achieves 100% automatic SSR and 87.5% human SSR on SD V2.1 with CLIP similarity 0.698, while random sampling baseline gets 0%.
- Prompt-space search on SD V2.1 reaches 59.8% human SSR versus 6.4% for greedy LLaMA baseline; e.g., 'cat fighting fish' yields no cat in 97% of images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adding residual connection and approximate gradients for latent-space search instead of standard PGD | SD V2.1 latent space search · SD V2.1 latent search | SSR(H): 12.5% to 80.7% | Helped |
| gradient-guided search over text prompts instead of greedy LLaMA search with ViT score | SD V2.1 prompt space · SD V2.1 prompt search | SSR(H): 6.4% to 59.8% | Helped |
| gradient-guided adversarial token embedding optimization instead of random sampling | SD V2.1 token embedding space · SD V2.1 token embedding search | SSR(H): 0.0% to 87.5% | Helped |
| adding edge-based (Canny) classifier to discriminator ensemble (ablation) | SAGE discriminator on SD V2.1 · SD V2.1 latent search, human eval | true negative rate / SSR(H): true negative rate 32.5% to 77.5% | Helped |
| adding residual connection at denoising steps (1 to 40 steps varied) (ablation) | latent-space search ablation over denoising step · latent search, step sweep | SSR(A): 100% for steps 1-30, 86% at step 40 | Mixed |
Surprising findings
- Adversarial token embeddings overwrite arbitrary prompts yet barely change CLIP similarity (0.742 vs 0.546 for the target), suggesting the diffusion process itself is attacked.
- Some latent regions with non-outlier Gaussian samples produce distorted images regardless of the text prompt.
Limitations
- Automatic evaluation relies on classifier ensembles that have their own false negatives; human verification needed.
- Latent and prompt searches are computationally heavy and required engineering tricks (FP16, parameter freezing) for baselines on A100 GPUs.
Experimental setup
Datasets: ImageNet-1k
Benchmarks: ImageNet-1k ImageNet-C
Metrics: search success rate failure generation rate non-Gaussian rate CLIP similarity
Baselines: PGD random sampling greedy search with LLaMA
Builds on: projected gradient descent CLIP LLaMA PoseExaminer
Closest papers by meaning
- On Discrete Prompt Optimization for Diffusion Models (ICML 2024 · similarity 0.88)
- FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges (NeurIPS 2025 · similarity 0.88)
- Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models (NeurIPS 2024 · similarity 0.87)
- CONFORM: Contrast is All You Need for High-Fidelity Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.87)
- Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts (ICML 2024 · similarity 0.87)
- The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.