AI Paper Atlas

Research map / Probabilistic and scientific modeling / Variational and generative sampling

Estimating High Order Gradients of the Data Distribution by Denoising

Chenlin Meng, Yang Song, Wenzhe Li, Stefano Ermon

NeurIPS 2021 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Score-based generative modeling analysis · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Directly estimate the second-order score with D₂SM instead of differentiating the DSM score modelSecond-order score estimation on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01MSE: 43.80±0.012 -> 1.76±0.038Improved
Increase D₂SM low-rank parameterization from rank 15 to rank 30 (ablation)D₂SM on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01MSE: 7.12± 0.319 -> 1.76±0.038Improved
Increase D₂SM low-rank parameterization from rank 20 to rank 30 (ablation)D₂SM on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01MSE: 5.24±0.065 -> 1.76±0.038Improved
Directly estimate the second-order score with D₂SM instead of differentiating the DSM-VR score modelDiagonal second-order score estimation on mixture of logistics · Mixture of logistics diagonal estimation (50-d, 20 mixtures), σ=0.01MSE: 26.41±0.55 -> 18.43± 0.11Improved
Directly estimate the second-order score with D₂SM instead of differentiating the DSM-VR score modelDiagonal second-order score estimation on mixture of logistics · Mixture of logistics diagonal estimation (80-d, 20 mixtures), σ=0.01MSE: 32.80± 0.34 -> 21.68± 0.18Improved
Use D₂SM rank 20 instead of automatic differentiationSecond-order score estimation, 100-d MLP · 100-d test-time estimationtime: 32100 ± 156 μs -> 380 ± 7.9 μsImproved
Use D₂SM rank 20 instead of automatic differentiationSecond-order score estimation, 784-d U-Net · 784-d test-time estimationtime: 34600 ± 194 ms -> 67.9 ± 1.93 msImproved
Increase low-rank parameterization from rank 20 to rank 1000 (ablation)D₂SM timing on 100-d MLP · 100-d test-time estimationtime: 380 ± 7.9 μs -> 1840 ± 97.1 μsHurt
Use Ozaki discretization with second-order score information instead of Langevin dynamicsSampling on synthetic datasets · Synthetic Dataset 1ESS: 21.81 -> 28.89Improved
Use Ozaki discretization with second-order score information instead of Langevin dynamicsSampling on synthetic datasets · Synthetic Dataset 2ESS: 26.33 -> 46.57Improved
Apply variance reduction to DSM rather than using standard DSM (ablation)Second-order score estimation by differentiating the DSM score model · Multivariate normal (100-d), σ=0.01MSE: 43.80±0.012 -> 9.40±0.049Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy