Research map / Probabilistic and scientific modeling / Variational and generative sampling
Dropout with Expectation-linear Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add expectation-linearization regularization to standard dropout | Six MNIST network architectures · MNIST test | classification error (%): 1.23$\pm$0.03 -> 1.07$\pm$0.02; 1.19$\pm$0.02 -> 1.03$\pm$0.02; 1.05$\pm$0.03 -> 0.98$\pm$0.03; 1.07$\pm$0.02 -> 0.94$\p | Helped |
| Add expectation-linearization regularization to standard dropout | 3 conv + 2 dense, relu + max-norm · CIFAR-10 test | classification error (%): 12.82\% to 12.20\% | Helped |
| Add expectation-linearization regularization to standard dropout | 3 conv + 2 dense, relu + max-norm · CIFAR-100 test | classification error (%): 37.22\% to 36.25\% | Helped |
| Use expectation-linearization regularization instead of MC dropout inference | AllConv · CIFAR-10 test | classification error (%): 10.58$\pm$0.21 -> 10.86$\pm$0.08 | Hurt |
| Use expectation-linearization regularization instead of MC dropout inference | AllConv · CIFAR-100 test | classification error (%): 34.43$\pm$0.25 -> 35.10$\pm$0.13 | Hurt |
| Add expectation-linearization regularization instead of standard dropout | AllConv · CIFAR-10 test | classification error (%): 11.18$\pm$0.11 -> 10.86$\pm$0.08 | Helped |
| Add expectation-linearization regularization instead of standard dropout | AllConv · CIFAR-100 test | classification error (%): 35.50$\pm$0.23 -> 35.10$\pm$0.13 | Helped |
| Increase the expectation-linearization regularization constant λ (ablation) | Standard dropout with EL · CIFAR-10 and CIFAR-100 | empirical expectation-linearization risk | Helped |
| Increase the expectation-linearization regularization constant λ (ablation) | Standard dropout with EL · MNIST, CIFAR-10, and CIFAR-100 | test error | Mixed |
Closest papers by meaning
- R-Drop: Regularized Dropout for Neural Networks (NeurIPS 2021 · similarity 0.92)
- Demystifying Dropout (ICML 2019 · similarity 0.91)
- Dropout Reduces Underfitting (ICML 2023 · similarity 0.91)
- Jumpout : Improved Dropout for Deep Neural Networks with ReLUs (ICML 2019 · similarity 0.91)
- Guided Dropout (AAAI 2019 · similarity 0.90)
- Shakeout: A New Regularized Deep Neural Network Training Scheme (AAAI 2016 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.