AI Paper Atlas

Research map / Learning theory and algorithms / Sample complexity

Gradient-Based Feature Learning under Structured Data

Alireza Mousavi-Hosseini, Denny Wu, Taiji Suzuki, Murat A Erdogdu

NeurIPS 2023 · Learning theory and algorithms › Sample complexity · approach: Sample complexity of learning distributions · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use covariance-aware normalization instead of spherical covariance-agnostic trainingSingle-neuron gradient flow under spiked covariance · Spiked covariance aligned with target, moderate spiketarget-direction alignment: \sup_{t \geq 0}\abs{\langle\boldsymbol{w}^t,\boldsymbol{u}\rangle} \lesssim 1/\sqrt{d} -> \langle\overline{\boldsymbol{wImproved
Use spherical covariance-agnostic flow rather than covariance-aware normalization (ablation)ReLU single neuron · Spiked covariance aligned with target, moderate spiketarget-direction alignment: \sup_{t \geq 0}\abs{\langle\boldsymbol{w}^t}{\boldsymbol{u}\rangle} \lesssim 1/\sqrt{d}Hurt
Increase spike magnitude in the regime 2r_1 < r_2 < 1Normalized gradient flow · Spiked covariancesample complexity: d^{1+2r_2}\left(d^{(s-1)(1-r_2)} \lor \varepsilon^{-2}\right)Improved
Increase spike-target alignment in the intermediate regimeNormalized gradient flow · Spiked covariance, r_1 < r_2 < 2r_1sample complexity: d^{1 + 2r_2}\left(d^{(s-1)(1-2(r_2-r_1))}\lor\varepsilon^{-2}\right)Improved
Precondition the normalized dynamics by inverse empirical covarianceNormalized gradient flow · General covariance structuresample complexity: n \gtrsim d\varkappa(\boldsymbol{\Sigma})^2\left\{\langle\overline{\boldsymbol{w}}^0,\overline{\boldsymbol{u}}\rangle^{2Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy