Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Dropout as a Structured Shrinkage Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rank-based tail-adaptive importance weights instead of lower-bound uniform weights | MC dropout · UCI regression benchmarks | test RMSE: best RMSE in five out of seven data sets | Helped |
| Use rank-based tail-adaptive importance weights instead of regular importance weights | MC dropout · UCI regression benchmarks | test log-likelihood: best log-likelihoods are achieved by regular importance sampling (five out of seven) | Hurt |
| Use rank-based tail-adaptive importance weights instead of lower-bound weights | MC dropout · Boston | test RMSE: 2.80 \scriptstyle{\pm .19} -> 2.369 \scriptstyle{\pm .22} | Helped |
| Use rank-based tail-adaptive importance weights instead of lower-bound weights | MC dropout · Concrete | test RMSE: 4.81 \scriptstyle{\pm .14} -> 3.935 \scriptstyle{\pm .35} | Helped |
| Use rank-based tail-adaptive importance weights instead of lower-bound weights | MC dropout · Energy | test RMSE: 1.09 \scriptstyle{\pm .05} -> 0.828 \scriptstyle{\pm .05} | Helped |
| Use rank-based tail-adaptive importance weights instead of lower-bound weights | MC dropout · Power | test RMSE: 4.00 \scriptstyle{\pm .04} -> 3.286 \scriptstyle{\pm .08 } | Hurt |
| Use rank-based tail-adaptive importance weights instead of regular importance weights | MC dropout · Boston | test RMSE: 2.450 \scriptstyle{\pm .25 } -> 2.369 \scriptstyle{\pm .22} | Helped |
| Use rank-based tail-adaptive importance weights instead of regular importance weights | MC dropout · Concrete | test log-likelihood: -2.888 \scriptstyle{\pm .08} -> -2.940 \scriptstyle{\pm .13} | Hurt |
| Use rank-based tail-adaptive importance weights instead of regular importance weights | MC dropout · Energy | test log-likelihood: -1.339 \scriptstyle{\pm .06} -> -1.349 \scriptstyle{\pm .04} | Hurt |
| Use rank-based tail-adaptive importance weights instead of regular importance weights | MC dropout · Power | test RMSE: 3.094 \scriptstyle{\pm .08} -> 3.286 \scriptstyle{\pm .08 } | Hurt |
| Apply noise in the hierarchical rather than expanded parametrization (ablation) | MC dropout / multiplicative-noise model · Moderate to strong shrinkage | MAP-bound tightness | Hurt |
| Add an importance-weighted objective rather than the usual Monte Carlo lower bound | MC dropout · Noise-marginalized likelihood | lower-bound tightness | Helped |
Closest papers by meaning
- On the Regularization Properties of Structured Dropout (CVPR 2020 · similarity 0.90)
- Joint Inference for Neural Network Depth and Dropout Regularization (NeurIPS 2021 · similarity 0.90)
- Variational Dropout Sparsifies Deep Neural Networks (ICML 2017 · similarity 0.90)
- Structured Bayesian Pruning via Log-Normal Multiplicative Noise (NeurIPS 2017 · similarity 0.90)
- Dropout with Expectation-linear Regularization (ICLR 2017 · similarity 0.90)
- Stochastic Modified Equations and Dynamics of Dropout Algorithm (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.