Research map / Optimization / Implicit bias and generalization
Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard dropout training with IWSGD using multiple dropout masks | Wide ResNet (depth=28, dropout=0.5) · CIFAR-10 test | test error: 3.88 (0.15) -> 3.55 (0.11) | Helped |
| Increase IWSGD sample count from 4 to 8 (ablation) | Wide ResNet (depth=28, dropout=0.5) · CIFAR-10 test | test error: 3.58 (0.05) -> 3.55 (0.11) | Helped |
| Replace standard dropout training with IWSGD using multiple dropout masks | Wide ResNet (depth=28, dropout=0.5) · CIFAR-100 test | test error: 19.12 (0.24) -> 17.63 (0.13) | Helped |
| Increase IWSGD sample count from 4 to 8 (ablation) | Wide ResNet (depth=28, dropout=0.5) · CIFAR-100 test | test error: 18.01 (0.16) -> 17.63 (0.13) | Helped |
| Train for 4 times more iterations instead of using IWSGD (ablation) | Wide ResNet (depth=28, dropout=0.5) · CIFAR-10 test | test error: 3.88 (0.15) -> 4.48 (0.15) | Hurt |
| Train for 4 times more iterations instead of using IWSGD (ablation) | Wide ResNet (depth=28, dropout=0.5) · CIFAR-100 test | test error: 19.12 (0.24) -> 20.70 (0.19) | Hurt |
| Replace standard dropout training with IWSGD using multiple masks | SAN with 2-layer LSTM · VQA test-dev, Open-Ended All | accuracy: 60.19 -> 60.41 | Helped |
| Increase IWSGD sample count from 5 to 8 (ablation) | SAN with 2-layer LSTM · VQA test-dev, Open-Ended All | accuracy: 60.31 -> 60.41 | Helped |
| Replace standard dropout training with IWSGD | SAN with 2-layer LSTM · VQA test-dev, Open-Ended Num | accuracy: 36.74 -> 34.70 | Hurt |
| Replace standard dropout training with IWSGD using 5 masks | Google-NIC · MSCOCO test | BLEU-4: 26.8 -> 27.5 | Helped |
| Replace standard dropout training with IWSGD using 5 masks | Google-NIC · MSCOCO test | METEOR: 22.6 -> 22.9 | Helped |
| Replace standard dropout training with IWSGD using 5 masks | Google-NIC · MSCOCO test | CIDEr: 82.2 -> 83.6 | Helped |
Closest papers by meaning
- On the Inherent Regularization Effects of Noise Injection During Training (ICML 2021 · similarity 0.90)
- Noise Injection Node Regularization for Robust Learning (ICLR 2023 · similarity 0.90)
- Ghost Noise for Regularizing Deep Neural Networks (AAAI 2024 · similarity 0.89)
- Dropout Reduces Underfitting (ICML 2023 · similarity 0.89)
- Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee (ICLR 2020 · similarity 0.88)
- Shakeout: A New Regularized Deep Neural Network Training Scheme (AAAI 2016 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.