Research map / Image generation / Fast diffusion sampling

Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes

ICML 2025 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Frames diffusion-model memorization as sharpness of the log-density Hessian, justifies Wen's score-difference metric through this lens, proposes an earlier-step Hessian-upscaled detection metric, and introduces SAIL, which optimizes initial noise to reduce memorization in Stable Diffusion.

Problem

Memorization in diffusion models (replicating training samples) raises privacy concerns, and existing detection metrics lack theoretical grounding and are costly or only reliable at late steps; mitigation often degrades prompts or model internals.

Approach

Treat memorization as sharpness of the learned log density, measured by Hessian eigenvalues or their trace (approximated by score norm); show Wen's score-difference metric equals a Hessian-eigenvalue gap; upscale it with a Hessian-score product for early-step detection; and mitigate by optimizing the initial noise x_T to lower sharpness while keeping the prompt fixed.

Key results

• On SD v1.4 with a single sampling step and four generations, the proposed Hessian-upscaled metric reaches AUC 0.998 and TPR@1%FPR 0.982, matching Wen et al.'s five-step, 16-generation performance.
• On SD v2.0, the proposed metric attains AUC 0.991 without full-step sampling.
• SAIL improves SSCD (lower memorization) and CLIP scores over RTA, RNA, text-embedding and attention-based mitigation baselines on SD v1.4 and v2.0.

Measured effects reported in the paper

Limitations

• Theoretical results assume Gaussian distributions, applied to diffusion latents only at early and middle timesteps.
• SAIL requires optimization of x_T at inference time, adding overhead, and is evaluated only on prompts whose training images are available.
• Hessian estimates for Stable Diffusion rely on the pretrained model without second-order score matching training.

Experimental setup

Datasets: 2D mixture of Gaussians MNIST memorized prompts for SD v1.4 and v2.0 (Wen et al.) COCO Lexica Tuxemon GPT-4 generated prompts

Benchmarks: Stable Diffusion v1.4 memorized prompts Stable Diffusion v2.0 memorized prompts MNIST memorization

Metrics: AUC TPR@1%FPR SSCD CLIP score

Baselines: Wen et al. score-difference metric Carlini-style l2 tile detection attention-based detection Random Token Addition Random Number Addition text embedding optimization attention score adjustment

Builds on: Wen et al. memorization detection metric classifier-free guidance DDIM Arnoldi iteration second-order score matching

Closest papers by meaning

• Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion (ICML 2025 · similarity 0.91)
• On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.90)
• Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.89)
• How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.89)
• On the Memorization of Consistency Distillation for Diffusion Models (NeurIPS 2026 · similarity 0.89)
• Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.89)