Research map / Image generation / Fast diffusion sampling
What’s the score? Automated Denoising Score Matching for Nonlinear Diffusions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces local-DSM, a denoising score matching objective built from local transition increments with Taylor-linearized nonlinear drifts, enabling automated score training for nonlinear diffusions in generative modeling and statistical physics.
Problem
Score-based training of diffusions with nonlinear inference processes is hard: ISM needs noisy Hutchinson divergence estimates, and DSM needs transition kernel scores unavailable in closed form for nonlinear drifts.
Approach
Derive a local DSM ELBO using transitions q(y_t|y_s) over nearby times s<t, approximate those transitions by locally linearizing the drift via a first-order Taylor expansion (solved with matrix-factorization matrix exponentials), and choose time pairs (s,t) via a schedule so that the integral of gg^T over [s,t] is constant to control linearization error.
Key results
- On CIFAR-10 with a Logistic prior, local-DSM ELBO reaches <=3.568 BPD versus <=3.741 for the ISM ELBO; with MoG prior, <=3.496 vs <=3.637.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ISM ELBO training with local-DSM ELBO | Diffusion model with Logistic prior · CIFAR-10 | BPD: ≤ 3.741 ± 0.09 -> ≤ 3.568 ± 0.07 | Helped |
| Replace ISM ELBO training with local-DSM ELBO | Diffusion model with MoG prior · CIFAR-10 | BPD: ≤ 3.637 ± 0.14 -> ≤ 3.496 ± 0.11 | Helped |
| Add perceptual weighting to local-DSM (ablation) | Diffusion model with Logistic prior · CIFAR-10 | BPD: ≤ 3.568 ± 0.07 -> ≤ 3.561 ± 0.09 | No clear effect |
| Add perceptual weighting to local-DSM (ablation) | Diffusion model with MoG prior · CIFAR-10 | BPD: ≤ 3.496 ± 0.11 -> ≤ 3.503 ± 0.151 | No clear effect |
| Increase scheduled-pair λ from 0.01 to 0.05 (ablation) | Local-DSM with Logistic prior · CIFAR-10 | BPD: ≤ 3.566 ± 0.097 -> ≤ 3.422 ± 0.096 | Helped |
| Increase scheduled-pair λ from 0.01 to 0.05 (ablation) | Local-DSM with MoG prior · CIFAR-10 | BPD: ≤ 3.465 ± 0.1242 -> ≤ 3.354 ± 0.1879 | Helped |
| Increase scheduled-pair λ (ablation) | Local-DSM · CIFAR-10 | Gap between local-DSM and unbiased ISM BPD | Hurt |
| Replace ISM with local-DSM training | Score model for non-equilibrium stochastic dynamics · Non-equilibrium stochastic dynamics | MMD | Helped |
| Replace ISM with local-DSM training | Score model for interacting particle systems · Interacting particle systems | Sample variance fidelity | Helped |
Surprising findings
- Local-DSM ELBO gives lower BPD than the unbiased ISM ELBO even though the local approximation introduces error, and the gap grows with the schedule constant lambda
Limitations
- First-order Taylor linearization introduces approximation error that grows with the gap between s and t
Experimental setup
Datasets: CIFAR-10 2D checkerboard non-equilibrium stochastic dynamics (nonlinear x,v system) interacting particle system (N=5)
Benchmarks: CIFAR-10 2D checkerboard non-equilibrium stochastic dynamics interacting particle system
Metrics: bits per dim MMD sample variance
Baselines: ISM ELBO with Hutchinson trace estimator DSM with linear processes
Builds on: denoising score matching implicit score matching variational diffusion models
Closest papers by meaning
- Training Diffusion-based Generative Models with Limited Data (ICML 2025 · similarity 0.90)
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization (ICLR 2024 · similarity 0.90)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.90)
- Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.