Research map / Image generation / Fast diffusion sampling

Nonparametric Estimation of a Factorizable Density using Diffusion Models

NeurIPS 2026 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that diffusion-model density estimators with sparse weight-sharing neural networks adapt to factorizable densities (Bayesian networks / MRFs) and attain the minimax TV rate n^{-β/(2β+d)} up to logs.

Problem

Nonparametric density estimation of high-dimensional densities that factorize into low-dimensional components; existing estimators rarely adapt to this structure, and the theory of diffusion-model density estimators has not addressed it.

Approach

Treat a score-based diffusion model as an implicit density estimator and approximate its time-indexed score with sparse weight-sharing neural networks (CNN/RNN-like); the factorization lets the approximation exploit a low effective dimension d, giving the minimax TV rate up to logs without knowing the structure.

Key results

• Main theorem: E[d_TV(p0, p̂)] ≤ C n^{-β/(2β+d)} up to polylog factors, matching the lower bound n^{-β/(2β+d)} over factorizable Hölder densities, adapting to d without knowing the factorization.
• Approximation theorem: weight-sharing networks with O(m) distinct parameters approximate the score with squared error m^{-2β/d} (polylog factors) in the integrated loss.
• Small-scale simulations with D=5 show diffusion models perform reasonably at moderate n, compared against KDE and BOS baselines.

Surprising findings

• A very recent preprint (concurrent work) obtained essentially the same estimator and rate with fully connected networks, so the paper does not show a distinct theoretical advantage of weight-sharing architectures.
• The authors conjecture that vanilla score-matching-based density estimators (Langevin) are sub-optimal, with rate n^{-(β-1)/(2β+d)} for the score.

Limitations

• Assumes Hölder smoothness, bounded-below density on the cube (assumption L) and boundary smoothness (assumption B), which are technical and conflict with natural boundary behavior.
• Theory covers the ERM over a constructed weight-sharing class; practical DDPM/U-Net architectures differ and the estimator is not fully adaptive (depends on β/d).
• Simulations are small-scale (D=5, 260K-parameter models) and do not directly verify the theoretical rate; TV distance is not computed explicitly.

Experimental setup

Datasets: synthetic 5-dimensional copula-based distributions (independence, Gaussian AR(1) copula, Clayton copula)

Benchmarks: synthetic 5-dimensional factorizable distributions

Metrics: total variation distance (approximate)

Baselines: kernel density estimation BOS (deep learning density estimator) DDPM with fully connected networks vanilla score matching estimator

Builds on: DDPM score matching Chen et al. diffusion density estimation theory Hammersley-Clifford theorem

Closest papers by meaning

• Shallow diffusion networks provably learn hidden low-dimensional structure (ICLR 2025 · similarity 0.89)
• Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.88)
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.88)
• Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.88)
• On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models (ICLR 2024 · similarity 0.88)
• Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure (NeurIPS 2024 · similarity 0.88)