Research map / Architectures and efficiency / RNNs and normalization
How Does Topology Influence Gradient Propagation and Model Performance of Deep Networks With DenseNet-Type Skip Connections?
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NN-Mass rather than parameter count to characterize accuracy across architectures | DenseNet-type MLPs · MNIST | goodness-of-fit R² for accuracy: R^2=0.19 -> R^2=0.85 | Helped |
| Use NN-Mass rather than parameter count to predict initial Jacobian mean singular values | DenseNet-type MLPs · MNIST | goodness-of-fit R² for mean singular values: R^2=0.31 -> R^2=0.79 | Helped |
| Keep width and NN-Mass similar while varying depth and parameter count | DenseNet-type MLPs · MNIST | training convergence | No clear effect |
| Increase NN-Mass at similar parameter count | DenseNet-type MLPs · MNIST | training convergence | Helped |
| Use NN-Mass rather than parameter count to predict accuracy | ResNets and Wide-ResNets · ImageNet | goodness-of-fit R² for Top1 accuracy: R^2=0.64 -> R^2=0.87 | Helped |
| Use NN-Mass rather than parameter count to predict accuracy | MobileNet-v2 · ImageNet | goodness-of-fit R² for Top1 accuracy: R^2=0.74 -> R^2=0.96 | Helped |
| Compress MobileNetV2 (1.4) with NN-Mass-guided architecture design | MobileNetV2 (1.4) · ImageNet | Top1 accuracy: 73.3 -> 72.9 | Hurt |
| Compress MobileNetV2 (0.75) with NN-Mass-guided architecture design | MobileNetV2 (0.75) · ImageNet | Top1 accuracy: 67.9 -> 68.56 | Helped |
| Design a smaller DenseNet-type CNN using NN-Mass under parameter constraints | DenseNet-type CNN · CIFAR-10 | test accuracy: 97.02% -> 96.82% | Hurt |
Closest papers by meaning
- Skip Connections Eliminate Singularities (ICLR 2018 · similarity 0.89)
- Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again (NeurIPS 2022 · similarity 0.88)
- The Shattered Gradients Problem: If resnets are the answer, then what is the question? (ICML 2017 · similarity 0.87)
- Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (ICLR 2020 · similarity 0.87)
- Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities (AAAI 2025 · similarity 0.86)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.