Research map / Optimization / Implicit bias and generalization
Curvature-corrected learning dynamics in deep neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Full natural gradient descent (q=1) instead of gradient descent | deep linear networks · teacher-student linear task | generalization error | Hurt |
| Fractional curvature correction (sqrt-NGD, q=2) | deep linear networks · teacher-student linear task | generalization error | Helped |
| Block-diagonal approximation of NGD (NGD-d) (ablation) | NGD · deep linear networks | parameter dynamics stability | Hurt |
| Block-diagonal approximation of sqrt-NGD (sqrt-NGD-d) (ablation) | sqrt-NGD · deep linear networks | parameter dynamics stability | No clear effect |
| Full curvature correction (NGD) instead of SGD | deep linear networks · singular mode dynamics analysis | convergence speed of input-output map | Helped |
| Full curvature correction (NGD) instead of SGD | deep linear networks · singular mode dynamics analysis | parameter update stability at saddle points | Hurt |
| Fractional curvature correction (sqrt-NGD) instead of SGD/NGD | deep linear networks · singular mode dynamics analysis | parameter update speed and map nonlinearity | Mixed |
| Block-diagonal approximation of NGD (NGD-d) | NGD · deep linear networks | conservation of weight-path shape | Hurt |
| Block-diagonal approximation of sqrt-NGD (sqrt-NGD-d) | sqrt-NGD · deep linear networks | conservation of weight-path shape | No clear effect |
| Curvature correction (any of NGD, sqrt-NGD) instead of SGD | deep linear networks · parameter path | weight balance across layers | No clear effect |
Closest papers by meaning
- Stepping on the Edge: Curvature Aware Learning Rate Tuners (NeurIPS 2024 · similarity 0.90)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.89)
- Deconstructing the Goldilocks Zone of Neural Network Initialization (ICML 2024 · similarity 0.88)
- Global curvature for second-order optimization of neural networks (ICML 2025 · similarity 0.88)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.