Research map / Image generation / Text-to-image diffusion
Fantastic Targets for Concept Erasure in Diffusion Models and Where To Find Them
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that fixed-target concept erasure in diffusion models causes side effects on related concepts, and proposes Adaptive Guided Erasure (AGE), which learns per-concept target concepts via a min-max objective over a Gumbel-Softmax mixture of concept embeddings.
Problem
Concept erasure in text-to-image diffusion models maps each undesirable concept to a fixed generic target (e.g., empty prompt), which causes unintended side effects on benign related concepts.
Approach
Model the concept space as a graph of erasure impacts (finding locality: erasing a concept mainly affects nearby concepts), then adaptively select a per-concept target via a min-max optimization, parameterized as a Gumbel-Softmax mixture over the concept embedding matrix, that is not a synonym yet highly sensitive to parameter change.
Key results
- On Imagenette object erasure, AGE reaches 73.6% PSR-1 and 95.6% PSR-5 versus 55.2% and 79.9% for the best baseline AP, with FID 16.1 and CLIP 26.0.
- On nudity erasure, AGE achieves NER-0.5 of 1.53% versus 1.70% for AP and FID 14.20 versus 15.52 for AP.
- AGE gets ESR-1 98.1% and ESR-5 95.7%, slightly below UCE's 100% erasure rate, so erasure strength is not uniformly best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AGE instead of AP for object-related concept erasure | Stable Diffusion 1.4 · Imagenette object erasure | PSR-5: 79.9\pm2.8 -> 95.6\pm1.1 | Helped |
| Use AGE instead of UCE for object-related concept erasure | Stable Diffusion 1.4 · Imagenette object erasure | ESR-5: 100\pm0.0 -> 95.7\pm2.5 | Hurt |
| Use AGE instead of AP for nudity erasure | Stable Diffusion 1.4 · I2P nudity erasure | NER-0.5: 1.70 -> 1.53 | Helped |
| Use AGE instead of AP for artistic style erasure | Stable Diffusion 1.4 · Artistic style erasure | CLIP erasing: 21.57 -> 22.44 | Hurt |
| Use AGE instead of AP for artistic style preservation | Stable Diffusion 1.4 · Artistic style erasure | CLIP preserving: 30.13 -> 30.45 | Helped |
| Map the erased concept to a synonym (ablation) | Concept erasure target selection · NetFive | erasure effectiveness | Hurt |
| Map the erased concept to a closely related, non-synonym concept (ablation) | Concept erasure target selection · NetFive | preservation performance | Helped |
| Map the erased concept to a general concept (ablation) | Concept erasure target selection · NetFive | DS-5: 83\%, 91\%, and 78\% | Hurt |
| Map the erased concept to a semantically unrelated concept instead of a generic empty target (ablation) | Concept erasure target selection · NetFive | preservation performance | No clear effect |
Surprising findings
- Mapping to a synonym of the concept minimizes side effects but is the least effective at erasing it.
- Mapping to a general upper-level concept (e.g., Dog) does not necessarily preserve related concepts better than an empty prompt.
Limitations
- Erasure success rate (ESR) is slightly lower than the strongest baselines such as UCE and AP/CA/MACE.
- Gumbel-Softmax mixture requires embedding all concepts in a candidate set, which is computationally costly for large concept spaces; the paper restricts to top-k=100 nearest concepts.
- Artistic style evaluation relies on CLIP and LPIPS rather than a reliable detector, and the paper shows a clear erasure-preservation trade-off with AP.
Experimental setup
Datasets: ImageNet Imagenette COCO 30K I2P prompts NetFive
Benchmarks: NetFive Imagenette I2P nudity artistic style prompts
Metrics: DS-5 CS-5 ESR-1 ESR-5 PSR-1 PSR-5 FID CLIP score NER LPIPS
Baselines: AP ESD UCE CA MACE
Builds on: ESD UCE Concept Ablation MACE
Closest papers by meaning
- When Are Concepts Erased From Diffusion Models? (NeurIPS 2025 · similarity 0.93)
- Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation (NeurIPS 2024 · similarity 0.90)
- Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters (CVPR 2025 · similarity 0.90)
- One-dimensional Adapter to Rule Them All: Concepts Diffusion Models and Erasing Applications (CVPR 2024 · similarity 0.90)
- MACE: Mass Concept Erasure in Diffusion Models (CVPR 2024 · similarity 0.89)
- All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.