Research map / Image generation / Fast diffusion sampling
Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?
Marta Aparicio Rodriguez, Anastasia Borovykh, Grigorios A. Pavliotis, Daniel James Korchinski
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| raise sample log-likelihood (more common samples) | diffusion model on RHM data · RHM | time to memorization: negative linear relation between log-likelihood and time to memorization | Improved |
| increase Zipf exponent alpha (thinner tails) | diffusion model on RHM data · RHM, width 4096 | mean time to memorization: mean memorization time decreases with increasing alpha | Improved |
| move Zipf variation to more abstract level (smaller l_z) | diffusion model on RHM data · RHM, width 1024, alpha=2 | time to memorization: l_z=0 delays memorization versus l_z=3 | Hurt |
| increase model width | diffusion model on RHM data · RHM | memorization speed: 4096 model >95% memorization in 4000 epochs vs 256 model barely 10% after 100000 epochs | Improved |
| add Zipf variation at root level l_z=0 | diffusion model on RHM data · RHM | preferential memorization of common samples | No clear effect |
| decrease model size | diffusion model on RHM data · RHM | partial memorization of frequent rule: more pronounced in the width 256 model | Hurt |
| decrease model size (109M to 63M parameters) | DDPM on CelebA · CelebA 10,000 images | slop phase length: longer slop phase for 63M model | Hurt |
| early stopping at train/test loss divergence | diffusion model on RHM data · RHM, l_z=3 | overproduction of common subfeatures: test loss increase generally precedes KL increase | Improved |
Closest papers by meaning
- On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.93)
- Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.92)
- Understanding diffusion models requires rethinking (again) generalization (NeurIPS 2026 · similarity 0.91)
- Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion (ICML 2025 · similarity 0.90)
- A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective (NeurIPS 2025 · similarity 0.90)
- On the Memorization of Consistency Distillation for Diffusion Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
