Research map / Image generation / Autoregressive and latent generation

Disentangled Sparse Representations for Concept-Separated Diffusion Unlearning

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Latent diffusion with learned tokenizers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SAEParate, an SAE for text-to-image diffusion unlearning that adds a concept-aware contrastive loss on SAE latents and a GeLU-enhanced encoder to reduce shared features across concepts, reporting state-of-the-art results on UnlearnCanvas.

Problem

SAE-based concept unlearning in text-to-image diffusion models uses sparse reconstruction objectives that do not enforce concept-wise separation, so suppressing one concept's latent features also degrades non-target concepts that share them, especially in joint style-object unlearning.

Approach

Train the SAE with a supervised-contrastive loss on cross-attention-weighted, timestep-paired sample-level latents so concept clusters separate in latent space, add a GeLU hidden layer to the encoder for expressiveness, and use hard-negative reweighting for joint style-object concepts.

Key results

• Single-attribute unlearning average of 97.09% on UnlearnCanvas, versus 94.03% for the next best method SAeUron.
• Joint style-object unlearning total average of 93.70% versus 81.75% for SalUn (second best) and 76.93% for SAeUron.
• Object unlearning UA of 95.20% versus 78.82% for SAeUron; cluster rate 1.00 vs 0.93 for vanilla SAE.

Surprising findings

• Retraining SAeUron with matched SAE epochs did not improve it, so gains are attributed to the method rather than longer training.

Limitations

• Requires concept labels during SAE training.
• Evaluation limited to image-based diffusion unlearning.
• Joint setting relies on hand-tuned per-concept hyperparameters (tau_c, gamma_c) tuned on validation prompts.

Experimental setup

Datasets: UnlearnCanvas anchor prompts from CA

Benchmarks: UnlearnCanvas I2P

Metrics: UA IRA CRA FID SC OC UP cluster rate centroid margin

Baselines: ESD FMN UCE CA SalUn SEOT SPM EDiff SHS SAeUron

Builds on: SAeUron supervised contrastive learning TopK sparse autoencoder BatchTopK SAE UnlearnCanvas

Closest papers by meaning

• SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders (ICML 2025 · similarity 0.91)
• You Can’t Have It Both Ways: Concept Entanglement Limits Diffusion Model Unlearning (NeurIPS 2026 · similarity 0.90)
• Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient (AAAI 2025 · similarity 0.89)
• CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models (NeurIPS 2026 · similarity 0.89)
• CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Models (NeurIPS 2025 · similarity 0.89)
• Fine-Grained Erasure in Text-to-Image Diffusion-based Foundation Models (CVPR 2025 · similarity 0.89)