Research map / Image generation / Fast diffusion sampling

Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models

NeurIPS 2024 · Image generation › Fast diffusion sampling · approach: Diffusion models as perception tools · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces NeMo, a gradient-free method that localizes memorization of individual training images to a few (often single) neurons in Stable Diffusion's cross-attention value layers and deactivates them to stop verbatim replication while preserving image quality.

Problem

Diffusion models can memorize and reproduce training images verbatim, raising privacy and copyright concerns; existing mitigations act on inputs or require retraining and do not localize where memorization resides in the model.

Approach

Localize memorization down to individual neurons in cross-attention value layers by combining out-of-distribution z-score activation detection (initial candidate set) with a layer-wise and neuron-wise refinement using an SSIM-based memorization score on initial noise predictions; then deactivate the identified neurons to permanently mitigate memorization without retraining.

Key results

• Deactivating a median of 4 neurons for verbatim-memorized prompts cuts SSCD_Gen from 1.0 to 0.10 while keeping CLIP alignment at 0.31.
• Random deactivation of similar neuron counts leaves memorization unchanged (SSCD_Gen 0.999 for verbatim).
• Blocking up to 750 memorization neurons causes no significant FID/KID degradation on COCO; 28 verbatim prompts are memorized by a single neuron.

Measured effects reported in the paper

Surprising findings

• Single neurons, e.g. neuron #25 in the first cross-attention value layer, are responsible for verbatim memorization of multiple prompts.
• Positive scaling of memorization neuron activations only slightly reduces memorization, while full deactivation is needed.

Limitations

• Template memorization is harder to mitigate and sometimes requires deactivating over 100 neurons.
• Only evaluated on Stable Diffusion v1.4; newer models trained on deduplicated data showed no detectable memorization.

Experimental setup

Datasets: LAION-2B-en COCO

Benchmarks: COCO

Metrics: SSCD_Orig SSCD_Gen D_SSCD CLIP score alignment FID CLIP-FID KID

Baselines: Prompt embedding adjustment Attention scaling Adding random tokens Random neuron deactivation

Builds on: Stable Diffusion Wen et al. memorization detection SSCD classifier-free guidance

Closest papers by meaning

• On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.90)
• Exploring Local Memorization in Diffusion Models via Bright Ending Attention (ICLR 2025 · similarity 0.89)
• How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.88)
• Classifier-Free Guidance Inside the Attraction Basin May Cause Memorization (CVPR 2025 · similarity 0.88)
• Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.87)
• Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.87)