Research map / Image generation / Fast diffusion sampling

An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models

NeurIPS 2025 · Image generation › Fast diffusion sampling · approach: Diffusion theory and learned samplers · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Derives closed-form gradient-flow and probability-flow ODE dynamics for linear, deep linear, and linear convolutional diffusion denoisers, showing an inverse-variance spectral law in learning time, and tests it on MLP-UNets and CNN-UNets.

Problem

Understanding which parts of the data distribution diffusion models learn first during training and how architectural inductive bias (linear, deep linear, convolutional) shapes the evolution of the generated distribution.

Approach

Uses Gaussian equivalence to reduce the full-batch DSM loss of linear denoisers to ridge regression, solves gradient flow and the PF-ODE in closed form under commuting weights, and derives a power-law (roughly inverse-variance) relation between mode emergence time and target variance; convolutional weight sharing multiplies rates by N while local convolution changes the bias.

Key results

• For Gaussian data with MLP-UNet, emergence time scales as tau* ~ lambda^-alpha with alpha=1.08 (d=256) and 1.05-1.13 (d=512), close to the linear theory prediction of alpha≈1.
• On FFHQ32 with MLP-UNet, increasing-variance modes give exponent -0.48 (R^2=0.97, N=57) and decreasing-variance modes -0.35 (R^2=0.92, N=2914), showing a slightly attenuated but robust spectral bias.
• CNN-UNet shows essentially no spectral bias (effective exponent alpha≈0): many eigenmodes rise and decay near-simultaneously, and narrower networks (ch=4) show stronger ordering than wide ones (ch=128).

Measured effects reported in the paper

Surprising findings

• CNN-UNet learns many eigenmodes simultaneously, with spectral bias effectively absent, contrary to the MLP case and the linear theory.
• For patch convolution with small kernels (3x3), the decreasing-mode scaling exponent weakens and the increasing-mode exponent can flip sign.

Limitations

• Theory assumes linear denoisers, full-batch gradient flow, small or orthogonal initialization, and independent weights across noise scales.
• No closed-form analytic treatment of convolutional U-Net dynamics; the CNN results deviate from linear theory and are explained only heuristically.
• The CNN spectral-bias absence is attributed to width-to-channel ratio based on limited ablations.

Experimental setup

Datasets: FFHQ MNIST CIFAR-10 AFHQ32 AFHQv2 synthetic Gaussian

Benchmarks: FFHQ32 MNIST CIFAR-10 AFHQ32 AFHQv2

Metrics: mode variance along data eigenbasis emergence time power-law exponent alpha R^2

Baselines: one-layer linear denoiser two-layer symmetric linear denoiser full-width linear convolution patch linear convolution

Builds on: EDM Gaussian equivalence deep linear networks

Closest papers by meaning

• Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure (NeurIPS 2024 · similarity 0.90)
• Generalization through variance: how noise shapes inductive biases in diffusion models (ICLR 2025 · similarity 0.90)
• When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets (ICML 2025 · similarity 0.90)
• Generalization in diffusion models arises from geometry-adaptive harmonic representations (ICLR 2024 · similarity 0.90)
• Generalization Dynamics of Linear Diffusion Models (NeurIPS 2026 · similarity 0.90)
• Towards a Mechanistic Explanation of Diffusion Model Generalization (ICML 2025 · similarity 0.89)