Research map / Optimization / Implicit bias and generalization
On Lazy Training in Differentiable Programming
Lénaïc Chizat, Edouard Oyallon, Francis Bach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase model scale factor alpha (move toward lazy regime) | VGG-11 · CIFAR-10 | test accuracy | Hurt |
| increase initialization variance tau (lazy training) | two-layer ReLU network, teacher-student · synthetic 100-D | test loss | Hurt |
| use scaling 1/sqrt(m) instead of 1/m as width grows | two-layer ReLU network · synthetic d=100 | test error | Hurt |
| widen CNN trained in lazy regime | linearized VGG-11 · CIFAR-10 | test accuracy | Improved |
Closest papers by meaning
- Beyond Lazy Training for Over-parameterized Tensor Decomposition (NeurIPS 2020 · similarity 0.87)
- An Improved Analysis of Training Over-parameterized Deep Neural Networks (NeurIPS 2019 · similarity 0.87)
- Convergence Rates of Non-Convex Stochastic Gradient Descent Under a Generic Lojasiewicz Condition and Local Smoothness (ICML 2022 · similarity 0.87)
- Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks (NeurIPS 2019 · similarity 0.87)
- Data-Efficient Learning with Neural Programs (NeurIPS 2024 · similarity 0.85)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
