Research map / Image generation / Text-to-image diffusion

You Can’t Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearning

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Concept erasure in diffusion models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves a lower bound linking concept-region overlap (entanglement coefficient kappa) to unavoidable collateral damage in diffusion concept unlearning, and benchmarks 13 methods on a robustness-retention Pareto frontier.

Problem

Concept unlearning in text-to-image diffusion models faces a tension: methods that robustly erase a target concept degrade semantically related neighbor concepts, while methods preserving neighbors leak the concept under indirect prompts.

Approach

Formalizes concepts as cross-attention activation regions and defines an entanglement coefficient kappa (overlap of target region with other concepts' fattened regions); proves any epsilon-robust erasure incurs neighbor preservation error gamma >= kappa(1-epsilon)/L, so perfect erasure and perfect retention are impossible when kappa > 0.

Key results

• No method reaches both UA > 90 and NP > 80 on average; TRUST reaches UA 94.2 at NP 76.4, while STEREO gets UA 99.8 but NP only 18.0.
• STEREO's neighbor NP decreases monotonically with estimated kappa (castle kappa 0.27 to dog kappa 0.89; Spearman r_s = -1.0 across five concepts).
• ESD on SDXL shows NP decreasing with entanglement (90.9 for cat, 85.0 castle, 77.1 dog), and ESD on FLUX reduces NP to 35.6-40.5 while preserving controls.

Surprising findings

• SLD attains the lowest IRR (0.5) yet NP only 58.8, while SEOT preserves neighbors (NP 85.1) but leaks (IRR 24.5), so mechanism family does not determine the operating point.
• FLUX's T5 text encoder shows no measurable overlap for the five targets, suggesting entanglement in the denoiser need not be inherited from the text encoder.

Limitations

• Evaluation mainly on five object targets with Stable Diffusion v1.4; SDXL covers three concepts with one method and FLUX kappa scaling is untested.
• kappa-hat is a neighborhood-restricted empirical lower-bound proxy whose absolute value depends on probe layer, timestep and candidate pool; theorem constants (L, Delta) are effective and only estimated.
• Object-category concepts only; artistic styles, identities and harmful content are untested.

Experimental setup

Datasets: LAION captions (for correlated context terms)

Benchmarks: UA/IRR/IRA/NP/CP/DamageGap on five target concepts (dog, bear, horse, cat, castle)

Metrics: UA IRR IRA NP CP DamageGap

Baselines: UCE RECE ESD SalUn EDiff CPE TRUST AdvUnlearn STEREO SEOT SAeUron SLD

Builds on: ESD UCE RECE STEREO SAFREE

Closest papers by meaning

• CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models (NeurIPS 2026 · similarity 0.90)
• Disentangled Sparse Representations for Concept-Separated Diffusion Unlearning (NeurIPS 2026 · similarity 0.90)
• The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.90)
• When Are Concepts Erased From Diffusion Models? (NeurIPS 2025 · similarity 0.90)
• Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient (AAAI 2025 · similarity 0.89)
• CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models (NeurIPS 2025 · similarity 0.89)