Research map / Architectures and efficiency / RNNs and normalization
Adversarial Dropout for Recurrent Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adversarial dropout on recurrent connections (K=2) instead of FD | LSTM · sMNIST | test error rate: 0.720 ($\pm$0.061) -> 0.644 ($\pm$0.046) | Helped |
| Add adversarial dropout on recurrent connections (K=2) instead of FD | LSTM · pMNIST | test error rate: 5.121 ($\pm$0.121) -> 5.030 ($\pm$0.065) | Helped |
| Increase adversarial-mask search iterations from K=1 to K=2 (ablation) | Adversarial dropout on recurrent connections · sMNIST | test error rate: 0.705 ($\pm$0.047) -> 0.644 ($\pm$0.046) | Helped |
| Increase adversarial-mask search iterations from K=1 to K=2 (ablation) | Adversarial dropout on recurrent connections · pMNIST | test error rate: 5.046 ($\pm$0.067) -> 5.030 ($\pm$0.065) | Helped |
| Use recurrent-connection adversarial dropout rather than last-hidden-state adversarial dropout (ablation) | LSTM · IMDB | test error rate (%): 6.471 ($\pm$0.058) -> 6.005 ($\pm$0.043) | Helped |
| Use recurrent-connection adversarial dropout rather than last-hidden-state adversarial dropout (ablation) | LSTM · Elec | test error rate (%): 5.784 ($\pm$0.064) -> 5.603 ($\pm$0.049) | Helped |
| Increase adversarial-mask search iterations from K=1 to K=2 (ablation) | Recurrent-connection adversarial dropout · IMDB | test error rate (%): 6.155 ($\pm$0.053) -> 6.005 ($\pm$0.043) | Helped |
| Increase adversarial-mask search iterations from K=1 to K=2 (ablation) | Recurrent-connection adversarial dropout · Elec | test error rate (%): 5.622 ($\pm$0.051) -> 5.603 ($\pm$0.049) | Helped |
| Add virtual adversarial training to recurrent adversarial dropout (K=2) (ablation) | LSTM · IMDB | test error rate (%): 6.005 ($\pm$0.043) -> 5.687 ($\pm$0.068) | Helped |
| Add virtual adversarial training to recurrent adversarial dropout (K=2) (ablation) | LSTM · Elec | test error rate (%): 5.603 ($\pm$0.049) -> 5.621 ($\pm$0.058) | No clear effect |
| Add adversarial dropout to AWD-LSTM | AWD-LSTM · PTB test | perplexity: 56.8 -> 56.4 | Helped |
| Add adversarial dropout to AWD-LSTM | AWD-LSTM · WikiText-2 test | perplexity: 64.1 -> 63.4 | Helped |
Closest papers by meaning
- Fraternal Dropout (ICLR 2018 · similarity 0.93)
- Adversarial Dropout for Supervised and Semi-Supervised Learning (AAAI 2018 · similarity 0.93)
- Structured in Space, Randomized in Time: Leveraging Dropout in RNNs for Efficient Training (NeurIPS 2021 · similarity 0.92)
- A Theoretically Grounded Application of Dropout in Recurrent Neural Networks (NeurIPS 2016 · similarity 0.91)
- R-Drop: Regularized Dropout for Neural Networks (NeurIPS 2021 · similarity 0.89)
- Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs (NeurIPS 2022 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.