Research map / Probabilistic and scientific modeling / Dynamical systems
Dynamics of neural scaling laws in random feature regression with powerlaw-distributed kernel eigenvalues
Jakob Kramp, Javed Lindner, Moritz Helias
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase Langevin noise temperature (larger 2/beta) (ablation) | random feature regression with power-law spectrum · teacher-student simulation, P=N=100 | test error | Hurt |
| train longer under noise (no early stopping) (ablation) | Langevin regression dynamics · teacher-student simulation | test error | Hurt |
| add early stopping at t* = (beta/2) * alpha/(1+alpha) | Langevin regression dynamics · noisy training | test error | Improved |
| increase L2 regularization 1/(g beta) (ablation) | Langevin regression dynamics · high fixed noise, P=N=100 | test error at long times | Improved |
| increase L2 regularization 1/(g beta) (ablation) | Langevin regression dynamics · high fixed noise, P=N=100 | minimum test error and its location | No clear effect |
| increase number of training samples P (ablation) | Langevin regression dynamics · power-law spectrum | bias and variance error: bias ~ (Pt)^(-alpha/(1+alpha)); variance ~ P^(-alpha/(1+alpha)) | Improved |
| vary P and N in optimal stopping time (ablation) | optimal stopping time t* · P=N=100 simulation | optimal stopping time | No clear effect |
Closest papers by meaning
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.92)
- Dynamical Decoupling of Generalization and Overfitting in Large Two-Layer Networks (NeurIPS 2025 · similarity 0.91)
- A Dynamical Model of Neural Scaling Laws (ICML 2024 · similarity 0.90)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.90)
- Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks (ICML 2025 · similarity 0.90)
- Generalization Dynamics of Linear Diffusion Models (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
