Research map / Optimization / Implicit bias and generalization
Beyond the Edge of Stability via Two-step Gradient Updates
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the teacher-student network beyond EoS with η=2.6 instead of η=0.5 | 16-neuron ReLU teacher-student network · 16-neuron teacher-student | flatness of the reached minimum | Helped |
| Use η=2.6 instead of η=0.5 | 16-neuron ReLU teacher-student network · 16-neuron teacher-student | weight-to-output ratio across neurons: k²=1.21 | Helped |
| Start near an unbalanced minimum and train with learning rate 1.02× EoS threshold | Quasi-symmetric matrix factorization · Quasi-symmetric matrix factorization | balance of top singular values: about 6.1 and 5.3 | Helped |
| Increase ησ₁² from (1,1.23) to (1.23,1.28) | Matrix factorization with generic initialization · Matrix factorization | orbit period: (1,1.23) to (1.23,1.28) | Mixed |
| Increase ησ₁² beyond 1.28 | Matrix factorization with generic initialization · Matrix factorization | dynamical stability: >1.28 | Hurt |
| Use learning rates beyond EoS | Two-layer single-neuron ReLU network · Population-loss teacher-student learning | alignment with teacher neuron: (d,1.1d] | Helped |
| Decay the learning rate from 2.6 to 0.5 after 100 epochs | 16-neuron ReLU teacher-student network · 16-neuron teacher-student | flatness of reached minimum: 100 epochs | Helped |
| Use η in (1,1.5) for the asymmetric scalar-factorization problem | f(x,y)=½(xy−1)² · Two-dimensional scalar factorization | parameter balance: (1,1.5) | Helped |
| Use η=1.25 rather than η=1.05 | f(x,y)=½(xy−1)² · Two-dimensional scalar factorization | orbit period: 1.05; 1.25 | Mixed |
| Use a learning rate beyond EoS rather than η=0.1 | ReLU MLPs on MNIST · MNIST, 3-, 4-, and 5-layer MLPs | alignment of gradient with top Hessian eigenvector | Helped |
Closest papers by meaning
- Convergence Rates for Gradient Descent on the Edge of Stability for Overparametrised Least Squares (NeurIPS 2025 · similarity 0.93)
- Classification at the Edge of Stability: Unifying Self-Stabilization and Convergence Rates (NeurIPS 2026 · similarity 0.93)
- On the Unstable Convergence Regime of Gradient Descent (AAAI 2024 · similarity 0.91)
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias (NeurIPS 2025 · similarity 0.90)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.90)
- A Minimalist Example of Edge-of-Stability and Progressive Sharpening (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.