Research map / Architectures and efficiency / RNNs and normalization
Residual Networks Behave Like Ensembles of Relatively Shallow Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| delete any single layer at test time (ablation) | VGG (15 layers) · CIFAR-10 | error | Hurt |
| delete a single residual module at test time (ablation) | ResNet-110 · CIFAR-10 | error | No clear effect |
| delete increasing numbers of residual modules (ablation) | ResNet-110 · CIFAR-10 | error: error remains around 0.2 after deleting 10 | Hurt |
| delete 20 modules (ablation) | ResNet-110 · CIFAR-10 | error | Hurt |
| reorder blocks by swapping k random pairs (ablation) | ResNet-110 · CIFAR-10 | error vs Kendall Tau | Hurt |
| train with only effective paths (23 of 54 modules per batch) (ablation) | ResNet-110 · CIFAR-10 | test error (%): 6.10 -> 5.96 | No clear effect |
| train with stochastic depth (ablation) | ResNet-110 · CIFAR-10 layer deletion | resilience | Helped |
| sample path lengths and measure gradient (ablation) | ResNet 54 blocks · CIFAR-10 | gradient norm | Hurt |
Closest papers by meaning
- Residual Alignment: Uncovering the Mechanisms of Residual Networks (NeurIPS 2023 · similarity 0.89)
- Multi-level Residual Networks from Dynamical Systems View (ICLR 2018 · similarity 0.88)
- Residual Connections Encourage Iterative Inference (ICLR 2018 · similarity 0.88)
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.85)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.85)
- Transformative or Conservative? Conservation laws for ResNets and Transformers (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.