Research map / Optimization / Implicit bias and generalization
Are ResNets Provably Better than Linear Predictors?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train an arbitrarily deep nonlinear residual unit with a linear output layer | Residual network · General data distribution and convex smooth loss | objective value at local minima | Helped |
| Add a direct skip connection from the input to the output | SGD-trained network · Bounded convex parameter domain | objective value of one iterate: O((bl+r√log(1/δ))/√T) | Helped |
| Add L2 regularization to the residual network objective | Scalar residual-network example · Squared loss, ε=1 | objective landscape | Hurt |
| Use a hard constraint on the parameter domain | Scalar residual-network example · Squared loss | objective landscape | Hurt |
Closest papers by meaning
- Are deep ResNets provably better than linear predictors? (NeurIPS 2019 · similarity 0.95)
- Residual Connections Encourage Iterative Inference (ICLR 2018 · similarity 0.91)
- Residual Alignment: Uncovering the Mechanisms of Residual Networks (NeurIPS 2023 · similarity 0.91)
- Towards Understanding the Importance of Shortcut Connections in Residual Networks (NeurIPS 2019 · similarity 0.91)
- On the Global Convergence of Training Deep Linear ResNets (ICLR 2020 · similarity 0.90)
- Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.