Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Vadam instead of BBVI | Bayesian neural network regression · UCI benchmarks | Test RMSE and test log-likelihood | Mixed |
| Use MC-Dropout instead of Vadam | Bayesian neural network regression · UCI benchmarks | Test RMSE | Mixed |
| Use VOGN with minibatch size 1 instead of Vadam | Bayesian logistic regression · USPS-3vs5 | Symmetric KL distance to MF-Exact | Helped |
| Decrease Vadam minibatch size (ablation) | Vadam · Breast-Cancer | Symmetric KL distance to MF-Exact | Helped |
| Set Vadam minibatch size to 1 (ablation) | Vadam · Breast-Cancer | Symmetric KL distance to MF-Exact | Helped |
| Use Vadam with minibatch size 1 rather than VOGN-1 (ablation) | Vadam · Breast-Cancer | Symmetric KL distance to MF-Exact | Mixed |
| Replace GGN Hessian approximation with gradient-magnitude approximation (ablation) | VOGN-derived Vprop/Vadam · Bayesian logistic regression | Posterior approximation quality | Hurt |
| Use weight perturbation for exploration | Vadam and VadaGrad · Half-Cheetah | Cumulative reward | Helped |
| Use Vadam or VadaGrad instead of Adam-based SGD exploration methods | Half-Cheetah · Half-Cheetah | Cumulative reward | Mixed |
| Use VOGN or Vadam instead of BBVI | Bayesian neural network · UCI benchmarks | Convergence speed | Helped |
Closest papers by meaning
- A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD (ICML 2025 · similarity 0.88)
- Understanding the Generalization of Stochastic Gradient Adam in Learning Neural Networks (NeurIPS 2025 · similarity 0.88)
- MARS: Unleashing the Power of Variance Reduction for Training Large Models (ICML 2025 · similarity 0.87)
- On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions (NeurIPS 2024 · similarity 0.87)
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.87)
- Variational Learning is Effective for Large Deep Networks (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.