Research map / Image generation / Text-to-image diffusion
Towards Memorization-Free Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Anti-Memorization Guidance (AMG), an inference-time framework with three guidance terms (despecification, caption deduplication, dissimilarity) and adaptive memorization detection that eliminates memorized generations from pretrained DDPMs and Stable Diffusion with minor FID/CLIP cost.
Problem
Pretrained diffusion models memorize and regurgitate training images (pixel-level and object-level), creating litigation and copyright risks for users of these models.
Approach
Training-free guidance that modifies the epsilon prediction at inference using three terms tied to the causes of memorization (overly specific prompts, duplicated images, duplicated captions), activated adaptively by a similarity threshold with a parabolic schedule, so that only likely-memorized steps are altered.
Key results
- On LAION5B text-conditional generation with Stable Diffusion v1.4, AMG (Main) reduces the fraction of images with SSCD above 0.5 from 7.07% (Gspe+Gdup variant) / 44.85% baseline Top5% context to 0.00%, with FID 99.12 vs 106.41 and CLIP 26.98 vs 28.04 for the baseline.
- On unconditional CIFAR-10 with iDDPM, AMG (Strong) reduces the fraction of images with nL2 below 1.4 from 0.93% to 0.00% while FID improves from 7.44 to 6.98.
- Ablation on LAION5B shows removing dissimilarity guidance (Gsim) leaves 31.62% of generations flagged as memorized, while the full model reaches 0.00%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the full AMG guidance framework | Stable Diffusion · LAION5B text-conditional | % > 0.5 SSCD: 44.85 -> 0.00 | Helped |
| apply the full AMG guidance framework | Stable Diffusion · LAION5B text-conditional | FID: 106.41 -> 99.12 | Helped |
| apply the full AMG guidance framework | Stable Diffusion · LAION5B text-conditional | CLIP: 28.04 -> 26.98 | Hurt |
| apply the full AMG guidance framework | iDDPM · CIFAR-10 unconditional | % < 1.4 nL2: 0.93 -> 0.00 | Helped |
| apply the full AMG guidance framework | iDDPM · CIFAR-10 class-conditional | % < 1.4 nL2: 1.53 -> 0.00 | Helped |
| remove dissimilarity guidance (Gsim) (ablation) | AMG · LAION5B text-conditional | % > 0.5 SSCD: 0.00 -> 31.62 | Hurt |
| remove dissimilarity guidance (Gsim) (ablation) | AMG · LAION5B text-conditional | FID: 99.12 -> 87.10 | Helped |
| remove caption deduplication guidance (Gdup) (ablation) | AMG · LAION5B text-conditional | FID: 99.12 -> 119.13 | Hurt |
| remove despecification guidance (Gspe) (ablation) | AMG · LAION5B text-conditional | FID: 99.12 -> 120.48 | Hurt |
| remove conditional guidance (ablation) | AMG · CIFAR-10 unconditional | FID: 7.25 -> 257.27 | Hurt |
| replace the parabolic schedule with a constant schedule (ablation) | AMG · CIFAR-10 unconditional | % < 1.4 nL2: 0.00 -> 0.04 | Hurt |
Surprising findings
- Applying guidance indiscriminately without adaptive detection eliminates memorization but raises CIFAR-10 FID from 7.44 to 257.27.
- Removing text conditioning (unconditional/class-conditional generation) appears to reduce memorization but does not fully eliminate it, and AMG-style guidance mainly affects coarse structure in early steps.
Limitations
- Gsim is needed to guarantee elimination and carries a quality/text-alignment cost that grows with guidance scale.
- Memorization detection relies on nearest-neighbor search over training data, which requires access to the training set (e.g., LAION5B index), and thresholds vary by dataset.
- Baselines were reimplemented by the authors, so comparisons depend on their settings.
Experimental setup
Datasets: LAION5B CIFAR-10 ImageNet LSUN Bedroom LAION-10k
Benchmarks: LAION5B CIFAR-10
Metrics: SSCD similarity (Top1, Top5%) percentage of images above SSCD 0.5 nL2 similarity FID CLIP score
Baselines: Stable Diffusion v1.4 Concept ablation GNI RT CWR RNA iDDPM with deduplicated retraining
Builds on: classifier-free guidance classifier guidance DDIM Improved DDPM
Closest papers by meaning
- Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.92)
- How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.92)
- Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.91)
- Exploring Local Memorization in Diffusion Models via Bright Ending Attention (ICLR 2025 · similarity 0.90)
- Classifier-Free Guidance Inside the Attraction Basin May Cause Memorization (CVPR 2025 · similarity 0.90)
- On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.