Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Variational Bayesian dropout: pitfalls and fixes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a log-uniform prior over weights | Bayesian neural networks with continuous, nonzero likelihood at zero · Posterior inference | posterior propriety | Hurt |
| Use a truncated log-uniform prior over an expanding log-weight interval | Bayesian logistic regression · Posterior inference | posterior moment stability | Hurt |
| Use the variational objective with a log-uniform prior | Variational Gaussian dropout · Neural network weight fitting | empirical performance | Mixed |
| Use a singular dropout approximating distribution with a continuous prior under standard VI | Variational Bernoulli or Gaussian dropout · Posterior approximation | KL objective definedness | Hurt |
| Replace Dirac masses with shrinking Gaussian noise or discretise prior and posterior on a shared refining grid | Variational Bernoulli dropout · Singular posterior approximation | limiting objective agreement | Helped |
| Use QKL to approximate a full-rank Gaussian with a degenerate Gaussian | QKL · Gaussian approximation | QKL optimum | Helped |
| Optimise QKL over prior hyperparameters | QKL-based approximate inference · Prior hyperparameter estimation | hyperparameter estimation reliability | Hurt |
Closest papers by meaning
- Structured Dropout Variational Inference for Bayesian Neural Networks (NeurIPS 2021 · similarity 0.92)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.88)
- Generalized Variational Inference via Optimal Transport (AAAI 2024 · similarity 0.88)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.88)
- Stochastic Modified Equations and Dynamics of Dropout Algorithm (ICLR 2024 · similarity 0.87)
- Bayesian Online Natural Gradient (BONG) (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.