Research map / Architectures and efficiency / RNNs and normalization
Residual Connections Encourage Iterative Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Share top residual blocks naively during training (ablation) | Resnet-110 · CIFAR-10 and CIFAR-100 | validation performance | Hurt |
| Use unshared BatchNorm statistics, gamma, and beta with shared top blocks (ablation) | Shared Resnet · CIFAR-10 | test error: 6.99 -> 6.62 | Helped |
| Use unshared BatchNorm statistics, gamma, and beta with shared top blocks (ablation) | Shared Resnet · CIFAR-100 | test error: 29.66 -> 29.94 | Hurt |
| Unroll the final residual block for 20 additional steps (ablation) | Resnet-110 · CIFAR-10 borderline test examples | accuracy: 43% -> 53% | Helped |
| Unroll the final residual block for 20 additional steps (ablation) | Resnet-110 · CIFAR-10 train set | loss: 0.0012 -> 0.001 | Helped |
| Add scaling to the final block while unrolling (ablation) | Resnet-110 · CIFAR-10 | cosine loss | Helped |
| Add unshared BatchNorm statistics to naive shared-block training (ablation) | Shared Resnet · Shared-block activation stability | activation explosion | Helped |
| Unshare BatchNorm gamma and beta in addition to its statistics (ablation) | Shared Resnet · Shared-block activation stability | activation explosion | Helped |
Closest papers by meaning
- Residual Alignment: Uncovering the Mechanisms of Residual Networks (NeurIPS 2023 · similarity 0.92)
- Are ResNets Provably Better than Linear Predictors? (NeurIPS 2018 · similarity 0.91)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.88)
- Transformative or Conservative? Conservation laws for ResNets and Transformers (ICML 2025 · similarity 0.87)
- Revisiting Residual Connections: Orthogonal Updates for Stable and Efficient Deep Networks (NeurIPS 2025 · similarity 0.87)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.