Research map / Architectures and efficiency / RNNs and normalization
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use learned structured dropout patterns instead of DropBlock | WRN-28-10 · CIFAR-10 | accuracy: 96.2 ± 0.07 -> 96.8 ± 0.09 | Helped |
| use learned structured dropout patterns instead of DropBlock | ResNet-50 · ImageNet | top-1 accuracy: 78.3 -> 78.7 | Helped |
| use learned structured dropout patterns instead of Stochastic Depth | EfficientNet · ImageNet | top-1 accuracy: 76.8 -> 77.5; 80.2 -> 80.9; 82.5 -> 83.1; 84.1 -> 84.7 | Helped |
| add RandAugment to AutoDropout | WRN-28-10 · CIFAR-10 | accuracy: 97.9 ± 0.06 | Helped |
| add AutoDropout to UDA | WRN-28-2 · CIFAR-10-4K | accuracy: 94.9 ± 0.12 -> 95.8 ± 0.04 | Helped |
| add AutoDropout to UDA | ResNet-50 · ImageNet-10% | top-1 accuracy: 68.8 -> 72.9 | Helped |
| use the learned dropout pattern | Transformer-XL · Penn Treebank | test perplexity: 56.0 -> 54.9 | Helped |
| transfer the Penn Treebank dropout pattern | Transformer-XL · WikiText-2 | test perplexity: 61.4 -> 59.9 | Helped |
| transfer the Penn Treebank dropout pattern | Transformer · IWSLT-14 De-En | BLEU: 34.4 -> 35.8 | Helped |
| transfer the Penn Treebank dropout pattern | Transformer · WMT-14 En-Fr | BLEU: 38.1 -> 40.0; 1.9 BLEU | Helped |
| remove dropout patterns from the selected ResNet-50 locations (ablation) | ResNet-50 · ImageNet | top-1 accuracy: about 1% | Hurt |
| replace learned layer-specific Transformer patterns with Variational Dropout in all layers (ablation) | Transformer-XL · Penn Treebank | validation perplexity: 59.8; 1.7 point higher | Hurt |
Closest papers by meaning
- DropBlock: A regularization method for convolutional networks (NeurIPS 2018 · similarity 0.90)
- On the Regularization Properties of Structured Dropout (CVPR 2020 · similarity 0.90)
- Dropout distillation (ICML 2016 · similarity 0.88)
- Beyond Random Masking: When Dropout meets Graph Convolutional Networks (ICLR 2025 · similarity 0.87)
- Dropout Reduces Underfitting (ICML 2023 · similarity 0.87)
- R-Drop: Regularized Dropout for Neural Networks (NeurIPS 2021 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.