Research map / Architectures and efficiency / RNNs and normalization
Weight Decay may matter more than µP for Learning Rate Transfer in Practice
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Closest papers by meaning
- On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling (NeurIPS 2025 · similarity 0.88)
- On the Provable Separation of Scales in Maximal Update Parameterization (ICML 2025 · similarity 0.88)
- Scaling Exponents Across Parameterizations and Optimizers (ICML 2024 · similarity 0.88)
- Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks (ICML 2024 · similarity 0.88)
- Why Do We Need Weight Decay in Modern Deep Learning? (NeurIPS 2024 · similarity 0.87)
- On the Weight Dynamics of Deep Normalized Networks (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.