Research map / Optimization / Implicit bias and generalization
The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add bias to a two-layer network (ablation) | Two-layer ReLU network · S¹, odd frequencies k ≥ 3 | representability and learning | Helped |
| Remove bias (ablation) | Two-layer ReLU network · S¹, odd frequencies k ≥ 3 | representability and learning | Hurt |
| Train on higher-frequency target functions rather than lower-frequency ones | Neural networks on S¹ · S¹, uniformly sampled circle | convergence-time growth exponent: O(k^{2.15}), O(k^{1.93}), O(k^{1.94}), O(k^{2.11}) | Hurt |
| Train on higher-frequency target functions rather than lower-frequency ones | Neural networks on S² · S², uniformly sampled sphere | convergence-time growth exponent: O(k^{2.74}), O(k^{2.87}), O(k^{3.13}) | Hurt |
| Use cross-entropy loss instead of the primary L₂ loss | Deep residual network · S¹, thresholded cosine classification | convergence-time growth exponent: O(K^{2.34}) | Mixed |
Closest papers by meaning
- On the Spectral Bias of Neural Networks (ICML 2019 · similarity 0.92)
- Frequency Bias in Neural Networks for Input of Non-Uniform Density (ICML 2020 · similarity 0.91)
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.88)
- Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learning (NeurIPS 2024 · similarity 0.87)
- Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws (NeurIPS 2025 · similarity 0.86)
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.