Research map / Optimization / Implicit bias and generalization
Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RMSprop diagonal preconditioner to SGLD | SGLD · MNIST CNN | test error: 0.71% -> 0.45% | Helped |
| add RMSprop diagonal preconditioner to SGLD | SGLD · MNIST FNN 1200-1200 | test error: 1.40% -> 1.24% | Helped |
| add RMSprop diagonal preconditioner to SGLD | SGLD · a9a BLR | test error: 14.85% -> 14.85% | No clear effect |
| sample posterior with Langevin noise instead of optimizing (RMSprop -> pSGLD) | RMSprop · MNIST FNN 1200-1200 | test error: 1.39% -> 1.24% | Helped |
| increase prior variance from 1 to 100 (ablation) | pSGLD · MNIST FNN 1200-1200 | test error: 1.24% -> 1.14% | Helped |
| distill sample ensemble into single network (ablation) | pSGLD (σ²=1) · MNIST FNN 1200-1200 | test error: 1.24% -> 1.41% | Hurt |
| add RMSprop preconditioner to SGLD | SGLD · 2D Gaussian simulation | reconstruction error / autocorrelation time | Helped |
Closest papers by meaning
- The promises and pitfalls of Stochastic Gradient Langevin Dynamics (NeurIPS 2018 · similarity 0.90)
- SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration (ICLR 2026 · similarity 0.90)
- The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank (AAAI 2024 · similarity 0.90)
- Preconditioned Langevin Dynamics with Score-based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems (NeurIPS 2025 · similarity 0.89)
- Sample as you Infer: Predictive Coding with Langevin Dynamics (ICML 2024 · similarity 0.88)
- High-dimensional Limit of SGD for Diagonal Linear Networks (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.