Research map / Optimization / Implicit bias and generalization
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Remove half of the neurons and rescale the remaining neurons | Two-layer MNIST network, N=12800 · MNIST | classification error increase: 0.35% | Hurt |
| Remove half of the neurons and rescale the remaining neurons | Three-layer MNIST network, N=12800 · MNIST | classification error increase: 0.4% | Hurt |
| Remove half of the neurons and rescale the remaining neurons | Three-layer CIFAR-10 network, N=12800 · CIFAR-10 | classification error increase: 1% | Hurt |
| Remove half of the neurons and rescale the remaining neurons | Four-layer CIFAR-10 network, N=12800 · CIFAR-10 | classification error increase: 3.6% | Hurt |
| Increase network width | Trained networks with half their neurons removed · MNIST and CIFAR-10 | performance degradation after dropout | Helped |
| Use a piecewise linear path to connect two SGD solutions | Two-layer MNIST and three-layer CIFAR-10 networks, N=3200 · MNIST and CIFAR-10 | classification error along the path | No clear effect |
Closest papers by meaning
- A Convergence Theory for Deep Learning via Over-Parameterization (ICML 2019 · similarity 0.89)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.89)
- Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup (NeurIPS 2019 · similarity 0.88)
- On Linear Stability of SGD and Input-Smoothness of Neural Networks (NeurIPS 2021 · similarity 0.88)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.88)
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.