Research map / Large language models / Language, safety and interpretability
Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RPG-RT, a black-box red-teaming framework that iteratively fine-tunes a Vicuna-7B prompt-modifier with DPO using rule-based preferences scored by a learned CLIP-decoupling scoring model, reaching higher ASR than baselines across 19 T2I systems and commercial APIs.
Problem
Red-teaming closed-source text-to-image systems with unknown and diverse defense mechanisms, where white-box methods need internal access and existing black-box methods assume knowledge of specific defenses; feedback from T2I systems is coarse and labeled, hard to use directly.
Approach
An LLM agent iteratively rewrites prompts, and its DPO fine-tuning uses preference pairs derived from rules over three feedback types (filter rejection, SFW, NSFW) plus a learned scoring model that decouples CLIP embeddings into a harmfulness score and a benign-semantics similarity score.
Key results
- On nudity across 19 T2I systems, RPG-RT reaches 80.98% ASR on text-match vs. 34.56% for the best baseline (FLIRT), and 40.35% on AdvUnlearn where the next-best baseline gets 1.93%.
- On commercial APIs with 10 nudity prompts, RPG-RT achieves 31.33% ASR on DALL-E 3 while all baselines are below 5%; 67.67% on Leonardo.ai vs. 22.67% best baseline.
- RPG-RT reports lowest PPL across methods (e.g., 13.67 on text-match vs. 1307 for SneakyPrompt).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iterative rule-based preference modeling and DPO fine-tuning | Vicuna-7B red-team agent · text-match | ASR: 34.56 -> 80.98 | Helped |
| Use iterative rule-based preference modeling and DPO fine-tuning | Vicuna-7B red-team agent · DALL-E 3 | ASR: 4.67 -> 31.33 | Helped |
| Use iterative rule-based preference modeling and DPO fine-tuning | Vicuna-7B red-team agent · SD v3, unseen prompts | ASR: 57.27 -> 96.77 | Helped |
| Remove the L_harm loss term (ablation) | RPG-RT scoring model · nudity red-teaming | ASR | Hurt |
| Remove the L_sim loss term (ablation) | RPG-RT scoring model · nudity red-teaming | ASR | Hurt |
| Remove the L_rec loss term (ablation) | RPG-RT scoring model · nudity red-teaming | ASR | Hurt |
| Remove the L_inno loss term (ablation) | RPG-RT scoring model · nudity red-teaming | semantic similarity | Hurt |
Surprising findings
- Ring-A-Bell achieves the best ASR on SLD-max (44.88 vs 41.15 for RPG-RT), the only setting where the proposed method is not top.
Limitations
- Training requires many queries to the target system (30 modifications per prompt over 10 rounds), which is costly for training.
- Relies on fixed external detectors (NudeNet, Q16, CASCo, celebrity classifier, OWL-ViT) to label outcomes, so attack success is bounded by detector accuracy.
- Online API evaluation uses only 10 prompts.
Experimental setup
Datasets: I2P GPT-4o-generated prompts for violence, discrimination, politicians, trademarks
Benchmarks: I2P nudity prompts violence, discrimination, politician, trademark prompts unseen I2P prompts (30-50% nudity) DALL-E 3, Leonardo.ai, SDXL online APIs Open-Sora text-to-video
Metrics: ASR FID CLIP similarity PPL
Baselines: MMA-Diffusion P4D-K P4D-N SneakyPrompt Ring-A-Bell FLIRT
Builds on: direct preference optimization CLIP Bradley-Terry model LoRA Ring-A-Bell SneakyPrompt FLIRT
Closest papers by meaning
- RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models (NeurIPS 2026 · similarity 0.91)
- ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024 · similarity 0.88)
- T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation (CVPR 2025 · similarity 0.87)
- Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts (ICML 2024 · similarity 0.87)
- Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints (AAAI 2025 · similarity 0.87)
- Preference Adaptive and Sequential Text-to-Image Generation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.