Research map / Image generation / Text-to-image diffusion
Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STG, a training-free method that guides text embeddings during diffusion sampling using a safety classifier evaluated on the expected denoised image, for safe text-to-image generation across nudity, violence, and artist-style removal.
Problem
Text-to-image diffusion models generate unsafe content (nudity, violence, copyrighted artist styles) from malicious prompts; existing training-free methods do not use intermediate or final sample states and lack theoretical grounding.
Approach
Apply gradient ascent on a time-dependent safety function of the expected clean image (via Tweedie-style estimate from the score network) to the text embedding at each sampling step, which by a first-order expansion yields a safe guidance term aligning the model likelihood with the safety objective.
Key results
- On Ring-A-Bell violence with FLUX, STG at tau=0.16 raises DSR from 0.11 to 0.70 with PP 0.60, while FID-1K stays near base (57.77 vs 56.58).
- On artist-style removal for Kelly McKernan, STG achieves LPIPS_e 0.58 and ACC_e 0.10, versus 0.42 and 0.25 for the best baseline SDG-like setting.
- STG achieves the lowest COCO FID-1K among compared methods, even below the base model, with a slight CLIP score drop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply STG (τ=0.18) rather than the base model | FLUX · Ring-A-Bell violence | DSR: 0.11 -> 0.53 | Helped |
| Apply STG (τ=0.18) rather than the base model | SD3 · Ring-A-Bell violence | DSR: 0.12 -> 0.54 | Helped |
| Apply STG (τ=0.18) rather than the base model | SDXL · Ring-A-Bell violence | DSR: 0.04 -> 0.50 | Helped |
| Apply STG (τ=0.18) rather than the base model | LCM · Ring-A-Bell violence | DSR: 0.02 -> 0.52 | Helped |
| Adjust text embeddings with STG | Stable Diffusion v1.4 · Remove Van Gogh style | LPIPS_e: 0.43 -> 0.46 | Helped |
| Adjust text embeddings with STG | Stable Diffusion v1.4 · Remove Kelly McKernan style | LPIPS_e: 0.42 -> 0.58 | Helped |
| Use FP16 inference (ablation) | STG on Stable Diffusion v1.4 · Nudity (Ring-A-Bell) | sampling time: 23.9 -> 14.0 | Helped |
| Use FP16 inference (ablation) | STG on Stable Diffusion v1.4 · Nudity (Ring-A-Bell) | GPU memory: 45.4 -> 22.8 | Helped |
| Use FP16 inference (ablation) | STG on Stable Diffusion v1.4 · Nudity (Ring-A-Bell) | DSR: 0.79 -> 0.79 | No clear effect |
| Set τ=0.40 instead of τ=0.15 (ρ=2.0) (ablation) | STG on Stable Diffusion v1.4 · Nudity (Ring-A-Bell) | DSR: 0.79 -> 0.88 | Helped |
| Set τ=0.40 instead of τ=0.15 (ρ=2.0) (ablation) | STG on Stable Diffusion v1.4 · Nudity (Ring-A-Bell) | PP: 0.90 -> 0.84 | Hurt |
Surprising findings
- STG attains lower FID-1K than the unmodified base model on COCO, despite safety guidance.
Limitations
- Added sampling cost from gradient computation over text embeddings (up to ~50.8 s/batch vs 15.8 base without FP16).
- Safety function approximation via Tweedie estimate and Taylor expansion lacks tight guarantees; approximation error bounded by Lipschitz constant.
- Hyperparameters (rho, tau, gamma) must be tuned per task and safety function.
Experimental setup
Datasets: Ring-A-Bell SneakyPrompt I2P COCO validation
Benchmarks: Ring-A-Bell SneakyPrompt Concept Inversion COCO artist-style removal prompts
Metrics: defense success rate prior preservation (1-LPIPS) FID-1K CLIP score LPIPS ACC
Baselines: UCE RECE SLD SAFREE Negative Prompt SDG ESD SPM DUO
Builds on: Universal Guidance Classifier Guidance DUO SAFREE Diffusion Adaptive Text Embedding
Closest papers by meaning
- Detect-and-Guide: Self-regulation of Diffusion Models for Safe Text-to-Image Generation via Guideline Token Optimization (CVPR 2025 · similarity 0.91)
- SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation (ICLR 2025 · similarity 0.90)
- Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Get What You Want, Not What You Don't: Image Content Suppression for Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.88)
- Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts (ICML 2024 · similarity 0.88)
- Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.