Research map / Learning theory and algorithms / Sample complexity
Gradient-Based Feature Learning under Structured Data
Alireza Mousavi-Hosseini, Denny Wu, Taiji Suzuki, Murat A Erdogdu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use covariance-aware normalization instead of spherical covariance-agnostic training | Single-neuron gradient flow under spiked covariance · Spiked covariance aligned with target, moderate spike | target-direction alignment: \sup_{t \geq 0}\abs{\langle\boldsymbol{w}^t,\boldsymbol{u}\rangle} \lesssim 1/\sqrt{d} -> \langle\overline{\boldsymbol{w | Improved |
| Use spherical covariance-agnostic flow rather than covariance-aware normalization (ablation) | ReLU single neuron · Spiked covariance aligned with target, moderate spike | target-direction alignment: \sup_{t \geq 0}\abs{\langle\boldsymbol{w}^t}{\boldsymbol{u}\rangle} \lesssim 1/\sqrt{d} | Hurt |
| Increase spike magnitude in the regime 2r_1 < r_2 < 1 | Normalized gradient flow · Spiked covariance | sample complexity: d^{1+2r_2}\left(d^{(s-1)(1-r_2)} \lor \varepsilon^{-2}\right) | Improved |
| Increase spike-target alignment in the intermediate regime | Normalized gradient flow · Spiked covariance, r_1 < r_2 < 2r_1 | sample complexity: d^{1 + 2r_2}\left(d^{(s-1)(1-2(r_2-r_1))}\lor\varepsilon^{-2}\right) | Improved |
| Precondition the normalized dynamics by inverse empirical covariance | Normalized gradient flow · General covariance structure | sample complexity: n \gtrsim d\varkappa(\boldsymbol{\Sigma})^2\left\{\langle\overline{\boldsymbol{w}}^0,\overline{\boldsymbol{u}}\rangle^{2 | Improved |
Closest papers by meaning
- Gradient-Weight Alignment as a Train-Time Proxy for Generalization in Classification Tasks (NeurIPS 2025 · similarity 0.87)
- Low Rank Gradients and Where to Find Them (NeurIPS 2025 · similarity 0.85)
- Can Neural Networks Achieve Optimal Computational-statistical Tradeoff? An Analysis on Single-Index Model (ICLR 2025 · similarity 0.85)
- From Information to Generative Exponent: Learning Rate Induces Phase Transitions in SGD (NeurIPS 2025 · similarity 0.85)
- Sample-Efficient Linear Representation Learning from Non-IID Non-Isotropic Data (ICLR 2024 · similarity 0.85)
- How does Gradient Descent Learn Features --- A Local Analysis for Regularized Two-Layer Neural Networks (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
