Research map / Architectures and efficiency / RNNs and normalization
Shakeout: A New Approach to Regularized Deep Neural Network Training
Guoliang Kang, Jun Li, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Shakeout instead of Dropout when data is scarce | FC network · MNIST (train size 500) | test error (%): 11.76 -> 10.81 | Improved |
| use Shakeout instead of Dropout when data is scarce | CNN · MNIST (train size 500) | test error (%): 6.16 -> 4.83 | Improved |
| use Shakeout instead of Dropout with plentiful data | FC network · MNIST (train size 8000) | test error (%): 2.95 -> 2.96 | No clear effect |
| use Shakeout instead of Dropout with plentiful data | CNN · MNIST (train size 8000) | test error (%): 1.23 -> 1.31 | Hurt |
| use Shakeout instead of Dropout | AlexFastNet · CIFAR-10 (train size 300) | test error (%): 65.34 -> 63.71 | Improved |
| use Shakeout instead of Dropout | AlexFastNet · CIFAR-10 (train size 5500) | test error (%): 36.01 -> 34.54 | Improved |
| use Shakeout instead of Dropout with full data | AlexFastNet · CIFAR-10 (train size 40000) | test error (%): 20.50 -> 20.56 | No clear effect |
| use Shakeout instead of Dropout | WRN-16-4 · CIFAR-10 (train size 40000) | test error (%): 9.32 -> 9.01 | Improved |
| add Dropout instead of standard BP | FC network · MNIST (train size 500) | test error (%): 13.66 -> 11.76 | Improved |
| use Shakeout instead of standard BP | WRN-16-4 · CIFAR-10 (train size 50000) | test error (%): 14.39 -> 7.97 | Improved |
| use Shakeout instead of Dropout on AlexNet FC7 and FC8 | AlexNet · ImageNet-2012 val | top-1 error (%): 42.99 -> 42.88 | No clear effect |
| use Shakeout instead of Dropout for weight pruning at ratio 96% | AlexNet · ImageNet-2012 val | relative accuracy loss: 22.50% | Improved |
Closest papers by meaning
- Shakeout: A New Regularized Deep Neural Network Training Scheme (AAAI 2016 · similarity 0.97)
- Dropout with Expectation-linear Regularization (ICLR 2017 · similarity 0.90)
- Dropout Reduces Underfitting (ICML 2023 · similarity 0.89)
- Surprising properties of dropout in deep networks (JMLR 2018 · similarity 0.88)
- Implicit Regularization of Dropout (TPAMI 2024 · similarity 0.88)
- Jumpout : Improved Dropout for Deep Neural Networks with ReLUs (ICML 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
