Research map / Image generation / Text-to-image diffusion

How Diffusion Models Memorize

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Inference-time T2I improvement · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes why diffusion models memorize training images, attributing memorization to classifier-free guidance causing overestimation of the training image in early denoising, which collapses latent diversity and locks trajectories onto the memorized sample.

Problem

Diffusion models memorize and reproduce training images, raising privacy and copyright concerns; prior work identified empirical factors but not the underlying mechanism of how memorization is triggered and propagated during denoising.

Approach

Memorization is driven by early-denoising overestimation of the training image induced by classifier-free guidance: the conditional noise prediction injects -x, and guidance scale g amplifies it linearly (x0 estimate ~ g·x), which collapses denoising trajectories. A latent decomposition x_t = w0 x + wT x_T quantifies deviations from the theoretical schedule that correlate with SSCD memorization sc

Key results

• Under memorization with guidance g=7.5, the squared l2 error of the x0 estimate at the first denoising step is larger than for non-memorized prompts, despite memorization being stronger.
• Excess contribution of the memorized image (sum of E[w0]-sqrt(alpha_bar)) correlates with SSCD score at Pearson 0.9203; overall trajectory deviation correlates at 0.9224.
• Trace of latent covariance after only 10 denoising steps correlates with SSCD at Pearson 0.7148.

Surprising findings

• Training loss is larger under memorization in early denoising when classifier-free guidance is applied, contradicting the view that memorization is simply overfitting.
• Unconditional noise predictions carry no information about the training image; the memorized content is injected solely by the conditional prediction.

Limitations

• Analysis relies on an approximation that the training loss is near zero under memorization and that alpha_bar_T is near zero.
• Experiments use a fixed set of 436 prompts from prior work and SSCD as the sole memorization metric.
• SD v2.1 has few memorized prompts due to deduplication, limiting quantitative conclusions there.

Experimental setup

Datasets: LAION-5B prompts (436 prompts from prior work, with paired training images)

Benchmarks: 436 memorization prompts from prior work (SD v1.4, SD v2.1, RealisticVision)

Metrics: SSCD score Pearson correlation cosine similarity squared l2 error of x0 estimate

Baselines: text-guidance magnitude detector transition-timestep guidance mitigation

Builds on: classifier-free guidance DDIM DDPM SSCD Carlini-style memorization detection via text-guidance magnitude

Closest papers by meaning

• Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.94)
• Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.92)
• Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion (ICML 2025 · similarity 0.92)
• Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.92)
• On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.92)
• Towards Memorization-Free Diffusion Models (CVPR 2024 · similarity 0.92)