Research map / Architectures and efficiency / RNNs and normalization
Surprising properties of dropout in deep networks
David P. Helmbold, Philip M. Long
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with dropout (p=1/2) instead of no regularization | ReLU network, K=5, depth 3, width 50 · Data (0,..,0)->0 and (1,..,1)->1 | count of negative weights: 584 more, 338 fewer, 78 equal out of 1000 runs | Mixed |
| rescale inputs by 0.5 to 1.5 under dropout vs weight decay | ReLU network, K=5, d=2, n=5 · Input scale experiment | average training loss | No clear effect |
Closest papers by meaning
- Demystifying Dropout (ICML 2019 · similarity 0.92)
- Dropout with Expectation-linear Regularization (ICLR 2017 · similarity 0.91)
- The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularization (NeurIPS 2021 · similarity 0.90)
- On the Implicit Bias of Dropout (ICML 2018 · similarity 0.90)
- On the Regularization Properties of Structured Dropout (CVPR 2020 · similarity 0.90)
- Stochastic Modified Equations and Dynamics of Dropout Algorithm (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
