Research map / Probabilistic and scientific modeling / Variational and generative sampling
Estimating High Order Gradients of the Data Distribution by Denoising
Chenlin Meng, Yang Song, Wenzhe Li, Stefano Ermon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Directly estimate the second-order score with D₂SM instead of differentiating the DSM score model | Second-order score estimation on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01 | MSE: 43.80±0.012 -> 1.76±0.038 | Improved |
| Increase D₂SM low-rank parameterization from rank 15 to rank 30 (ablation) | D₂SM on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01 | MSE: 7.12± 0.319 -> 1.76±0.038 | Improved |
| Increase D₂SM low-rank parameterization from rank 20 to rank 30 (ablation) | D₂SM on multivariate normal (100-d) · Multivariate normal (100-d), σ=0.01 | MSE: 5.24±0.065 -> 1.76±0.038 | Improved |
| Directly estimate the second-order score with D₂SM instead of differentiating the DSM-VR score model | Diagonal second-order score estimation on mixture of logistics · Mixture of logistics diagonal estimation (50-d, 20 mixtures), σ=0.01 | MSE: 26.41±0.55 -> 18.43± 0.11 | Improved |
| Directly estimate the second-order score with D₂SM instead of differentiating the DSM-VR score model | Diagonal second-order score estimation on mixture of logistics · Mixture of logistics diagonal estimation (80-d, 20 mixtures), σ=0.01 | MSE: 32.80± 0.34 -> 21.68± 0.18 | Improved |
| Use D₂SM rank 20 instead of automatic differentiation | Second-order score estimation, 100-d MLP · 100-d test-time estimation | time: 32100 ± 156 μs -> 380 ± 7.9 μs | Improved |
| Use D₂SM rank 20 instead of automatic differentiation | Second-order score estimation, 784-d U-Net · 784-d test-time estimation | time: 34600 ± 194 ms -> 67.9 ± 1.93 ms | Improved |
| Increase low-rank parameterization from rank 20 to rank 1000 (ablation) | D₂SM timing on 100-d MLP · 100-d test-time estimation | time: 380 ± 7.9 μs -> 1840 ± 97.1 μs | Hurt |
| Use Ozaki discretization with second-order score information instead of Langevin dynamics | Sampling on synthetic datasets · Synthetic Dataset 1 | ESS: 21.81 -> 28.89 | Improved |
| Use Ozaki discretization with second-order score information instead of Langevin dynamics | Sampling on synthetic datasets · Synthetic Dataset 2 | ESS: 26.33 -> 46.57 | Improved |
| Apply variance reduction to DSM rather than using standard DSM (ablation) | Second-order score estimation by differentiating the DSM score model · Multivariate normal (100-d), σ=0.01 | MSE: 43.80±0.012 -> 9.40±0.049 | Improved |
Closest papers by meaning
- On the Posterior Distribution in Denoising: Application to Uncertainty Quantification (ICLR 2024 · similarity 0.89)
- Sampling Binary Data by Denoising through Score Functions (ICML 2025 · similarity 0.89)
- Particle Denoising Diffusion Sampler (ICML 2024 · similarity 0.88)
- Beyond Scores: Proximal Diffusion Models (NeurIPS 2025 · similarity 0.87)
- Noise Conditional Variational Score Distillation (ICML 2025 · similarity 0.87)
- What’s the score? Automated Denoising Score Matching for Nonlinear Diffusions (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
