Research map / Architectures and efficiency / RNNs and normalization
Gradual DropIn of Layers to Train Very Deep Neural Networks
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Expand the 3-layer CIFAR-10 network to 11 layers and add gradual DropIn | CIFAR-10 classifier · CIFAR-10 test | accuracy (%): 81.4 -> 82.3 | Helped |
| Use DropIn length 8,000 instead of 16,000 (ablation) | 11-layer CIFAR-10 net + DropIn · CIFAR-10 test | accuracy (%): 82.3 -> 81.7 | Hurt |
| Use DropIn length 24,000 instead of 16,000 (ablation) | 11-layer CIFAR-10 net + DropIn · CIFAR-10 test | accuracy (%): 82.3 -> 82.3 | No clear effect |
| Expand AlexNet to 13 layers and add gradual DropIn | AlexNet · ImageNet validation | accuracy (%): 58.0 -> 62.2 | Helped |
| Use DropIn length 75,000 instead of 25,000 (ablation) | AlexNet 13 layers + DropIn · ImageNet validation | accuracy (%): 62.2 -> 62.1 | No clear effect |
| Use DropIn length 150,000 instead of 25,000 (ablation) | AlexNet 13 layers + DropIn · ImageNet validation | accuracy (%): 62.2 -> 60.8 | Hurt |
| Use DropIn length 300,000 instead of 25,000 (ablation) | AlexNet 13 layers + DropIn · ImageNet validation | accuracy (%): 62.2 -> 59.3 | Hurt |
| Add gradual DropIn while incrementally including added layers | Expanded LeNet on MNIST · MNIST | trainability | Helped |
| Add gradual DropIn to the expanded 11-layer architecture | CIFAR-10 classifier · CIFAR-10 | trainability | Helped |
| Add gradual DropIn to the expanded AlexNet architecture | AlexNet · ImageNet | trainability | Helped |
| Add gradual DropIn to VGG16 without shallow-network weight transfer | VGG16 · ImageNet | trainability | Helped |
Closest papers by meaning
- DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training (ICLR 2023 · similarity 0.88)
- DropNet: Reducing Neural Network Complexity via Iterative Pruning (ICML 2020 · similarity 0.88)
- Guided Dropout (AAAI 2019 · similarity 0.87)
- Dropout with Expectation-linear Regularization (ICLR 2017 · similarity 0.87)
- Dropout distillation (ICML 2016 · similarity 0.87)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.