Research map / Image generation / Fast diffusion sampling

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion

ICML 2025 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains diffusion models with DDPM loss at low noise levels and ambient score matching on a single noisy copy of the training set at high noise levels, reducing memorization without hurting FID.

Problem

Diffusion models memorize training data, especially with small datasets, and prior memorization-mitigation methods typically reduce image quality. The paper asks whether models can have both high generation quality and low memorization.

Approach

Split the diffusion time axis at a noise level t_n: use standard DDPM denoising loss for t <= t_n on clean data, and Ambient Score Matching on a fixed set of single noisy copies (created once at t_n) for t > t_n, so high-noise scores do not point to training points; theory argues memorization is only necessary at low noise (adapting Feldman's subpopulation model).

Key results

• On CIFAR-10 with 300 images, the method reaches FID 23.91 vs 25.1 for DDPM while reducing the fraction of samples with similarity >0.95 from 56.56% to 9.64%.
• For text-conditional Stable Diffusion fine-tuning, combining the method with W24 lowers similarity to 0.192 (from 0.208) and 95th-percentile similarity to 0.267 with FID 20.74 vs 21.44.
• Unconditional results show same-or-better FID than DDPM across CIFAR-10, FFHQ and ImageNet at 300/1k/3k images, with lower memorization that shrinks as dataset size grows.

Measured effects reported in the paper

Surprising findings

• DDPM-trained models are not Pareto optimal for small datasets: the method achieves equal or better FID with substantially lower memorization.
• Ambient-only training on noisy data (noise baseline) yields much worse FID, showing the high-frequency details must come from the clean low-noise part.

Limitations

• The method has no privacy guarantees or optimality properties, and the paper lacks an end-to-end theoretical analysis.
• Text-conditional memorization is not reduced by the method alone; it requires combination with existing text-mitigation strategies.
• The single parameter t_n must be tuned within a range; too low or too high reverts to memorization or DDPM behavior.

Experimental setup

Datasets: CIFAR-10 FFHQ ImageNet (tiny) LAION subset (10k image-text pairs)

Benchmarks: CIFAR-10 FFHQ ImageNet (tiny) LAION subset (text-conditional)

Metrics: FID DINOv2 nearest-neighbor similarity (memorization) CLIP score

Baselines: DDPM Masking (ambient inpainting) Noise-only ambient training Somepalli et al. (S23) Wen et al. (W24)

Builds on: Ambient Diffusion DDPM Feldman (2020) subpopulations model Somepalli et al. 2023 text-embedding corruption Wen et al. 2024 memorization mitigation

Closest papers by meaning

• On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.93)
• How Diffusion Models Memorize (NeurIPS 2026 · similarity 0.92)
• Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024 · similarity 0.92)
• Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.92)
• Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes (ICML 2025 · similarity 0.91)
• Evaluating the design space of diffusion-based generative models (NeurIPS 2024 · similarity 0.90)