Research map / Image generation / Fast diffusion sampling

Generalization Dynamics of Linear Diffusion Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes generalization of linear diffusion models trained on finite Gaussian data with power-law covariance spectra, using replica theory to show two regimes (N<d overfitting, N>d KL decays as d/N) and effects of hierarchy, regularization, early stopping, and noise vs. data prediction.

Problem

How finite sample size N interacts with hierarchical (power-law) covariance spectra to govern generalization and overfitting in diffusion models, which classical learning theory predicts requires sample sizes exponential in dimension.

Approach

Models diffusion denoisers as affine linear maps trained on empirical covariance, derives closed-form test/train loss gaps, KL divergence and training dynamics, and uses replica trick to average over training sets; shows hierarchy and regularization mitigate overfitting, and predicting data instead of noise reweights the test loss toward leading eigenmodes.

Key results

• For N<d the test-train gap is large due to nullspace of the empirical covariance; regularization and more hierarchical spectra reduce overfitting.
• For N>d the KL divergence to the true Gaussian decays approximately as d/(4N), independent of the covariance spectrum.
• Predicting data instead of noise reweights the test loss toward leading eigendirections, masking overfitting in subleading modes.

Surprising findings

• A highly hierarchical spectrum has no significant effect on the N at which the two overfitting regimes intersect.
• Predicting the data instead of noise yields a test loss that hides lacking variability in subleading eigendirections, making it less sensitive to overfitting.

Limitations

• Analysis restricted to linear (affine) denoisers and Gaussian data; nonlinear U-Nets differ especially in leading eigendirections at high SNR.
• Continuous-time approximations and neglect of finite-batch noise and discrete sampling corrections.
• Memorization behavior of linear models does not qualitatively match nonlinear models.

Experimental setup

Datasets: CIFAR-10 CelebA MNIST synthetic Gaussian with power-law spectrum

Benchmarks: CIFAR-10 CelebA MNIST

Metrics: test loss train loss KL divergence generalization gap Delta epsilon_N

Baselines: U-Net diffusion model

Builds on: linear diffusion models (prior analysis) replica trick denoising score matching

Closest papers by meaning

• Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure (NeurIPS 2024 · similarity 0.92)
• Generalization through variance: how noise shapes inductive biases in diffusion models (ICLR 2025 · similarity 0.92)
• Understanding diffusion models requires rethinking (again) generalization (NeurIPS 2026 · similarity 0.92)
• On the Edge of Memorization in Diffusion Models (NeurIPS 2025 · similarity 0.91)
• Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in Training (NeurIPS 2025 · similarity 0.91)
• Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifold (NeurIPS 2026 · similarity 0.90)