Research map / Probabilistic and scientific modeling / Dynamical systems
The Three Stages of Learning Dynamics in High-dimensional Kernel Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase gradient-flow training time | Kernel least-squares gradient flow · High-dimensional regression | learned function complexity | Helped |
| Continue training past the interpolation threshold | Empirical kernel model · High-dimensional regression | training error | Helped |
| Train through the second learning stage | Empirical kernel model · High-dimensional regression | test error relative to oracle error | No clear effect |
| Use a group-invariant kernel instead of a dot-product kernel | Kernel least-squares gradient flow · Invariant function estimation | test error | Helped |
| Use a group-invariant kernel instead of a dot-product kernel | Kernel least-squares gradient flow · Invariant function estimation | time to interpolate training data | Hurt |
| Use a dot-product kernel instead of a group-invariant kernel | Kernel least-squares gradient flow · Invariant function estimation | optimization speed | Helped |
| Use a dot-product kernel instead of a group-invariant kernel | Kernel least-squares gradient flow · Invariant function estimation | test error | Hurt |
| Use an invariant kernel, which is equivalent to augmenting the dataset with group transforms | Kernel least-squares · Group-invariant regression | effective sample size | Helped |
| Choose sample size at a special scaling or use a target that is a degree-s polynomial | Kernel learning dynamics · High-dimensional regression | duration of the second learning stage | Hurt |
| Place the degree-s target component in the orthogonal complement of the RKHS | Kernel learning dynamics · High-dimensional regression | presence of the third learning stage | Hurt |
Closest papers by meaning
- Three Mechanisms of Feature Learning in a Linear Network (ICLR 2025 · similarity 0.89)
- Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws (NeurIPS 2025 · similarity 0.87)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.87)
- Does SGD really happen in tiny subspaces? (ICLR 2025 · similarity 0.87)
- The Optimization Landscape of SGD Across the Feature Learning Strength (ICLR 2025 · similarity 0.87)
- Towards the Dynamics of a DNN Learning Symbolic Interactions (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.