Research map / Optimization / Implicit bias and generalization
Mitigating the Curse of Detail: Scaling Arguments for Feature Learning and Sample Complexity
Noa Rubin, Orit Davidovich, Zohar Ringel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| allow GP-to-specialization feature learning in layer 2 instead of lazy GP-GP | three-layer erf FCN (Hermite-3 target) · predicted sample complexity scale | variational energy: d^3 -> sqrt(N_2 d) | Improved |
| use specialization-magnetization pattern instead of GP-GP | three-layer erf FCN (Hermite-3 target) · predicted sample complexity scale | variational energy: d^3 -> (N_1 N_2 d)^{1/3} | Improved |
| increase first-layer width N_1 (specialization-magnetization vs GP-specialization) | three-layer erf FCN · sample complexity | P_*: N_1^{1/3} vs no N_1 dependence | Mixed |
| specialization pattern instead of GP in softmax attention head | softmax attention (cubic target) · predicted sample complexity | variational energy: L d^3 -> sqrt(H L d^3) | Improved |
| two-layer FCN with Hermite-3 target, proportional limit | two-layer erf FCN · sample complexity scaling in d | P_*: P_* = d | Improved |
Closest papers by meaning
- How Feature Learning Can Improve Neural Scaling Laws (ICLR 2025 · similarity 0.89)
- Understanding Visual Feature Reliance through the Lens of Complexity (NeurIPS 2024 · similarity 0.88)
- From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning (ICML 2025 · similarity 0.88)
- Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning (NeurIPS 2024 · similarity 0.88)
- Feature Learning Dynamics in Infinite-Depth Neural Networks (NeurIPS 2026 · similarity 0.88)
- Three Mechanisms of Feature Learning in a Linear Network (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
