Research map / Optimization / Implicit bias and generalization
SGD on Neural Networks Learns Functions of Increasing Complexity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| explain early SGD network with linear classifier instead of equal-accuracy random classifier | SGD-trained CNN · CIFAR-10 first 5 vs last 5 | performance correlation ratio: 0.02 -> 0.74 (table); text reports 0.31 -> 0.80 for CIFAR (iii) | Mixed |
| linear classifier instead of null model | SGD-trained CNN · Binary MNIST | performance correlation ratio: 0.52 -> 0.79 | Helped |
| linear classifier instead of null model | SGD-trained MLP · High-dimensional sinusoid | performance correlation ratio: 0.27 -> 0.74 | Helped |
| 6-layer CNN simple model instead of null model | ResNet-18 SGD · CIFAR-10 first 5 vs last 5 | performance correlation ratio: 0.40 -> 0.72 | Helped |
| initialize from bad (random-label-fitted) init instead of good SGD init | 3-layer MLP · 2D Gaussian with 10% label noise | population accuracy | Hurt |
Closest papers by meaning
- SGD with Large Step Sizes Learns Sparse Features (ICML 2023 · similarity 0.89)
- Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup (NeurIPS 2019 · similarity 0.89)
- The Three Stages of Learning Dynamics in High-dimensional Kernel Methods (ICLR 2022 · similarity 0.89)
- SGD Converges to Global Minimum in Deep Learning via Star-convex Path (ICLR 2019 · similarity 0.89)
- Emergence and scaling laws in SGD learning of shallow neural networks (NeurIPS 2025 · similarity 0.88)
- From Information to Generative Exponent: Learning Rate Induces Phase Transitions in SGD (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.