Research map / Image generation / Text-to-image diffusion
Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Semantic Surgery, a training-free zero-shot concept erasure method that subtracts a calibrated, presence-scaled concept direction from the text embedding before diffusion, with co-occurrence encoding and visual feedback for multi-concept and persistence cases.
Problem
Text-to-image diffusion models generate harmful or biased content; existing concept erasure methods either alter model parameters (losing locality and being static) or act token-wise or mid-diffusion, failing to achieve both completeness and locality.
Approach
Operate globally on the text embedding before diffusion: estimate concept presence via a sigmoid-calibrated cosine score (semantic biopsy), then subtract a scaled concept-direction vector computed from a co-occurrence prompt; a visual feedback loop reinforces erasure when concepts persist in generated images.
Key results
- Object erasure on CIFAR-10 reaches average H-score 93.58, versus 88.74 for Receler and 87.13 for MACE, with Acc_R of 2.00.
- I2P nudity erasure yields 1 nude instance versus 123 for MACE and 82 for SAFREE, with FID 12.2 on MS-COCO.
- Artistic style erasure reaches H_a 8.09 versus 5.99 for MACE, with FID-30K 14.04 matching SD v1.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Semantic Surgery rather than Receler | CIFAR-10 object erasure · CIFAR-10 object erasure (average) | H: 88.74 -> 93.58 | Helped |
| Use Semantic Surgery rather than Receler | CIFAR-10 object erasure · CIFAR-10 object erasure (average) | Acc_R: 10.00 -> 2.00 | Helped |
| Use Semantic Surgery rather than MACE | CIFAR-10 object erasure · CIFAR-10 object erasure (average) | Acc_L: 79.09 -> 85.56 | Helped |
| Use Semantic Surgery rather than SAFREE | I2P explicit content removal · I2P | Total NudeNet detections: 82 -> 1 | Helped |
| Use Semantic Surgery rather than SD v1.4 | MS-COCO 30k general image generation · MS-COCO 30k | FID: 14.04 -> 12.2 | Helped |
| Use Semantic Surgery rather than MACE | Artistic style erasure · Artistic style erasure (100 styles) | H_a: 5.99 -> 8.09 | Helped |
| Use Semantic Surgery rather than SD v1.4 | Artistic style erasure · MS-COCO 30k | FID-30K: 14.04 -> 14.04 | No clear effect |
| Use Semantic Surgery rather than MACE | Multi-celebrity erasure · 100 celebrities | H_c: ≈ 0.892 -> ≈ 0.965 | Helped |
| Use Semantic Surgery rather than MACE | Black-box RAB adversarial attack · RAB (380 prompts) | ASR: 3.95% -> 1.05% | Helped |
| Add visual feedback (ablation) | Semantic Surgery · CIFAR-10 object erasure (average) | H-score: 92.18 -> 93.58 | Helped |
| Add visual feedback (ablation) | Semantic Surgery · CIFAR-10 object erasure (average) | Acc_L: 87.38 -> 85.56 | Hurt |
| Add visual feedback (ablation) | Semantic Surgery · I2P explicit content removal | Total nude images: 47 -> 1 | Helped |
Surprising findings
- Visual feedback reduces I2P nude instances from 47 to 1 while FID improves to 12.2, below the SD v1.4 baseline of 14.04.
- The method's FID-30K on the style task matches SD v1.4 while MACE's FID is lower (12.71), so the FID claim is relative to the original model only.
Limitations
- Visual feedback depends on detector quality; false positives (e.g., truck as automobile) reduce locality.
- Hyperparameter beta is task-dependent and must be chosen from empirical alpha distributions.
Experimental setup
Datasets: CIFAR-10 I2P Image Synthesis Style Studies MACE celebrity dataset MS-COCO 30k RAB prompts UnlearnDiffAtk
Benchmarks: CIFAR-10 object erasure I2P Image Synthesis Style Studies (100 artists) MACE multi-celebrity erasure RAB UnlearnDiffAtk
Metrics: H-score Acc_E Acc_R Acc_L NudeNet instance count FID-30K CLIP-30K H_a H_c ASR
Baselines: SD v1.4 ESD-x ESD-u UCE AC Receler MACE SLD SAFREE
Builds on: activation engineering SLD SAFREE
Closest papers by meaning
- All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.93)
- MACE: Mass Concept Erasure in Diffusion Models (CVPR 2024 · similarity 0.92)
- Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation (CVPR 2025 · similarity 0.91)
- Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models (NeurIPS 2026 · similarity 0.91)
- Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters (CVPR 2025 · similarity 0.90)
- Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.