Research map / Image generation / Text-to-image diffusion

Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models

CVPR 2025 · Image generation › Text-to-image diffusion · approach: Concept erasure in diffusion models · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FADE, a LoRA-based (Mesh Modules) concept-erasure method for text-to-image diffusion models that uses a data-driven Concept Neighborhood to preserve semantically adjacent concepts while erasing a target, evaluated with a new ERB score.

Problem

Existing concept-unlearning methods for text-to-image diffusion models erase a target concept but also degrade semantically related (adjacent) concepts, a problem termed adjacency.

Approach

Builds an adjacency set of top-K most similar classes via cosine similarity of CLIP-style image-embedding means generated by the model, then trains LoRA-style Mesh Modules with an erasing loss, a null-guidance loss and an adjacency-preservation loss; also introduces the Erasing-Retention Balance (ERB) harmonic-mean metric.

Key results

• FADE achieves ERB of 95.97 on Welsh Springer Spaniel erasure vs. 4.68 for Receler and 33.34 for ESD in the fine-grained setting (Table 1).
• On ImageNet-1k, FADE reports at least 12% higher ERB than the next-best method SPM (Table 3 and text).
• On I2P nudity erasure, FADE achieves an erasure ratio change of 87.88% versus the SD v1.4 baseline, the highest among methods; FID on COCO30K is 13.86 vs 13.52 for FMN.

Measured effects reported in the paper

Surprising findings

• Receler reaches 100% erasure accuracy but near-zero adjacency accuracy (about 0.1–6%), showing erasure can fully destroy neighboring concepts.
• Ablation shows removing the guidance loss yields perfect erasure (100%) yet lowers ERB from 95.97 to 86.44.

Limitations

• Adjacency sets depend on similarity computed from the model's own generated images, so errors in the neighborhood propagate to the loss.
• Evaluation relies on classifiers fine-tuned on generated images and on a 40-person user study, limiting external validity.
• Theoretical k-NN-to-Naive-Bayes result requires conditional independence and asymptotic assumptions unlikely to hold for real image features.

Experimental setup

Datasets: Stanford Dogs Oxford Flowers CUB ImageNet-1k Imagenette I2P COCO30K

Benchmarks: Stanford Dogs Oxford Flowers CUB ImageNet-1k Imagenette I2P

Metrics: Erasing accuracy Adjacency accuracy ERB score Atar/Aothers accuracy NudeNet detection counts FID

Baselines: ESD Concept Ablation (CA) Forget-Me-Not (FMN) SPM Receler UCE

Builds on: ESD Concept Ablation SPM LoRA UCE

Closest papers by meaning

• Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient (AAAI 2025 · similarity 0.91)
• Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation (CVPR 2025 · similarity 0.91)
• All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.91)
• CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models (NeurIPS 2025 · similarity 0.91)
• Enabling Preference-driven Unlearning in Few-step Distilled Text-to-Image Diffusion Models (NeurIPS 2026 · similarity 0.90)
• The Illusion of Unlearning: The Unstable Nature of Machine Unlearning in Text-to-Image Diffusion Models (CVPR 2025 · similarity 0.90)