Research map / Image generation / Text-to-image diffusion
Ablating Concepts in Text-to-Image Diffusion Models
Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, Jun-Yan Zhu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune the full U-Net weights with the model-based ablation objective on memorized prompts | Stable Diffusion · Memorized images (8 prompts) | % samples with SSCD similarity >= 0.5: 60.1 -> 0.3 | Improved |
| ablate concepts with our method versus MSCOCO FID of the pretrained model | Stable Diffusion · MSCOCO FID | FID: 16.35 -> 16.99 ± 0.2 | No clear effect |
| use model-based objective (match model predictions) instead of loss maximization baseline (ablation) | Concept ablation, cross-attention fine-tuning · Instances (Grumpy Cat, Snoopy, Nemo, R2D2) | CLIP Score / accuracy on surrounding concepts | Improved |
| model-based variant instead of noise-based variant (ablation) | Concept ablation · Instances | convergence speed / CLIP metrics | Improved |
| fine-tune text embedding only versus cross-attention (ablation) | Concept ablation · Instances (spelling-mistake prompts) | robustness to spelling mistakes | Hurt |
| fine-tune full weights instead of other parameter subsets for memorized images (ablation) | Concept ablation · Memorized images | memorization removal | Improved |
| use reverse KL approximation instead of forward KL (ablation) | Concept ablation · Styles vs instances | CLIP metrics | Mixed |
| choose broader anchor category (British Shorthair Cat or Felidae) instead of Cat (ablation) | Grumpy Cat ablation · Grumpy Cat | qualitative samples | No clear effect |
| train on 1000 images instead of 200 (ablation) | Concept ablation · Surrounding concepts | CLIP metrics | No clear effect |
| increase training images from 1000 to 10k (ablation) | Concept ablation · Grumpy Cat | ablation performance | No clear effect |
| increase number of unique training prompts (10, 50 vs 200) (ablation) | Concept ablation · Grumpy Cat | convergence speed | Improved |
| replace ChatGPT with QLoRA for generating training prompts (ablation) | Grumpy Cat ablation · Grumpy Cat target and surrounding | CLIP Score: 0.639 -> 0.651 (target); 0.796 -> 0.801 (surrounding) | No clear effect |
Closest papers by meaning
- The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.90)
- Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models (CVPR 2025 · similarity 0.89)
- Growth Inhibitors for Suppressing Inappropriate Image Concepts in Diffusion Models (ICLR 2025 · similarity 0.89)
- MACE: Mass Concept Erasure in Diffusion Models (CVPR 2024 · similarity 0.89)
- All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.89)
- Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
