Research map / Image generation / Fast diffusion sampling
Locality in Image Diffusion Models Emerges from Data Statistics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that locality in image diffusion models stems from pixel correlations in the training data, not CNN inductive bias, and uses a Wiener-filter-derived binary mask to build an analytical denoiser that better matches deep diffusion models.
Problem
Prior work attributes generalization in image diffusion models to architectural locality (CNN inductive bias) and fits patch sizes to trained networks, leaving unexplained why locality arises and how it depends on data.
Approach
Locality of trained denoisers' sensitivity fields is approximated by the high-SNR principal-component projection of the optimal linear (Wiener) denoiser; this is computed analytically from data covariance and binarized to define per-pixel masks in a patch-based optimal denoiser, with no equivariance and no measurement from trained networks.
Key results
- On CIFAR-10 the proposed analytical model reaches r^2 0.589 vs 0.408 for Wiener filter and 0.303 for Kamb et al.
- On CelebA-HQ the proposed model reaches r^2 0.902 vs 0.831 for Kamb et al. (MSE 0.016 vs 0.028)
- On MNIST the proposed model reaches r^2 0.491, versus 0.469 for the Wiener filter and 0.402 for Kamb et al.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · CIFAR10 | r²: 0.408±0.092 -> 0.589±0.078 (Wiener); 0.303±0.126 -> 0.589±0.078 (Kamb) | Helped |
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · CIFAR10 | MSE: 0.032±0.004 -> 0.028±0.008 (Wiener); 0.065±0.017 -> 0.028±0.008 (Kamb) | Helped |
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · CelebA-HQ | r²: 0.818±0.039 -> 0.902±0.032 (Wiener); 0.831±0.073 -> 0.902±0.032 (Kamb) | Helped |
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · CelebA-HQ | MSE: 0.031±0.004 -> 0.016±0.006 (Wiener); 0.028±0.005 -> 0.016±0.006 (Kamb) | Helped |
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · MNIST | r²: 0.469±0.066 -> 0.491±0.051 (Wiener); 0.402±0.092 -> 0.491±0.051 (Kamb) | Helped |
| use dataset-statistics-derived sensitivity masks instead of the analytical baselines | analytical denoiser · MNIST | MSE: 0.161±0.014 -> 0.153±0.015 (Wiener); 0.188±0.039 -> 0.153±0.015 (Kamb) | Helped |
| set binarization threshold τ to 0.050 instead of 0.020 (ablation) | Ours · CIFAR10 | r²: 0.672 -> 0.773 | Helped |
| set binarization threshold τ to 0.070 instead of 0.020 (ablation) | Ours · CIFAR10 | r²: 0.672 -> 0.771 | Helped |
| set binarization threshold τ to 0.020 instead of 0.005 (ablation) | Ours · CelebA-HQ | r²: 0.786 -> 0.897 | Helped |
| set binarization threshold τ to 0.005 instead of 0.020 (ablation) | Ours · MNIST | r²: 0.418 -> 0.492 | Helped |
| set binarization threshold τ to 0.005 instead of 0.020 (ablation) | Ours · Fashion MNIST | r²: 0.406 -> 0.563 | Helped |
| set binarization threshold τ to 0.070 instead of 0.020 (ablation) | Ours · Fashion MNIST | r²: 0.406 -> -0.192 | Hurt |
Surprising findings
- A simple Wiener filter outperforms prior expert-crafted analytical denoisers based on optimal-denoiser modifications
- Introducing equivariance to the proposed model does not improve performance but increases inference time
Limitations
- Assumes locality fields are constant with respect to the input image
- Relies only on second-order (covariance) statistics, not higher-order structure learned by deep networks
- Focuses on simpler architectures (U-Net without self-attention)
Experimental setup
Datasets: CIFAR-10 CelebA-HQ AFHQv2 MNIST Fashion-MNIST
Benchmarks: CIFAR-10 CelebA-HQ MNIST
Metrics: r^2 MSE
Baselines: Optimal denoiser Wiener filter Kamb and Ganguli analytical model Another DDPM (different seed)
Builds on: optimal denoiser Kamb and Ganguli analytic diffusion Wiener filter DDPM
Closest papers by meaning
- Towards a Mechanistic Explanation of Diffusion Model Generalization (ICML 2025 · similarity 0.88)
- Unsupervised Region-Based Image Editing of Denoising Diffusion Models (AAAI 2025 · similarity 0.88)
- Feature Information Dynamics in Diffusion (NeurIPS 2026 · similarity 0.88)
- Ambient Diffusion Omni: Training Good Models with Bad Data (NeurIPS 2025 · similarity 0.87)
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs (ICLR 2024 · similarity 0.87)
- Generalization in diffusion models arises from geometry-adaptive harmonic representations (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.