Research map / Architectures and efficiency / RNNs and normalization
Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply LM-architecture (two-step linear multistep skip connection) to ResNet | ResNet-20/56/110/164 pre-act · CIFAR10 | test error (%): 8.75 -> 8.33 (20 layers); 6.97 -> 6.31 (56); 6.37 -> 6.16 (110); 5.46 -> 5.27 (164) | Helped |
| Apply LM-architecture to ResNet | ResNet-164 pre-act · CIFAR100 | test error (%): 24.33 -> 22.90 | Helped |
| Apply LM-architecture to ResNeXt-29 (16x64d, pre-act) | ResNeXt-29 16x64d · CIFAR100 | test error (%): 17.65 -> 16.79 | Helped |
| Apply LM-architecture to ResNet-50 and ResNet-101 | ResNet on ImageNet · ImageNet validation | top-1 error (%): ResNet-50 24.7 -> 23.8; ResNet-101 23.6 -> 22.6 | Helped |
| Add stochastic depth to LM-ResNet (ablation) | LM-ResNet-110 pre-act · CIFAR10 | test error (%): LM-ResNet-110 without SD 6.16 -> 4.80 with SD | Helped |
| Add stochastic depth to ResNet-56 pre-act | ResNet-56 pre-act · CIFAR10 | test error (%): 6.97 (ResNet-56 without SD, Table 1 original) ->5.55 with SD (own implementation, pre-act) | Helped |
| Apply LM-architecture plus stochastic depth to ResNet-56 (ablation) | ResNet-56 pre-act SD · CIFAR10 | test error (%): 5.55 -> 5.14 | Helped |
| Use LM-architecture with learned coefficients k_n (initialized from U[-0.1,0]) vs plain ResNet on CIFAR100 with deeper networks | LM-ResNet-110 vs ResNet-110 pre-act · CIFAR100 | test error (%): LM-ResNet-110 25.87 vs ResNet-110 not reported on CIFAR100 in table | No clear effect |
Closest papers by meaning
- Scaling Properties of Deep Residual Networks (ICML 2021 · similarity 0.88)
- Implicit regularization of deep residual networks towards neural ODEs (ICLR 2024 · similarity 0.86)
- Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation (AAAI 2025 · similarity 0.86)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.86)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.86)
- Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.