AI Paper Atlas

Research map / Optimization

Implicit bias and generalization: research map

1,070 papers on Implicit bias and generalization in Optimization, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 3 clusters and 12 approaches. The busiest year so far is 2026.

What it is

Neural networks have far more parameters than data points yet generalise. Part of the reason is that training itself prefers certain solutions (implicit bias). This theory topic studies those preferences: flat minima, the edge of stability, the neural tangent kernel and simplified networks.

Terms you will see on this page:

Implicit bias
the kind of solution an algorithm such as SGD tends to find without explicit regularisation.
Sharpness
how quickly the loss rises around a solution; flat minima are thought to generalise better.
Neural tangent kernel (NTK)
a kernel that describes very wide networks during training.

Start reading with the most cited papers here:

Within Optimization, its share held steady from 27.1% in 2023–24 to 26.7% in 2025–26 (235 → 261 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).

2016: 13162017: 26172018: 49182019: 84192020: 100202021: 116212022: 117222023: 138232024: 133242025: 142252026: 15226

Explore Implicit bias and generalization in the interactive map

Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.

What changed recently

Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):

Papers from 2025–2026 that are already cited most:

Approaches and key papers

Theory of wide and shallow networks · 479 papers
Keywords: relu, width, kernel

Approaches in this cluster:

Most cited and most cited since 2024:

SGD noise and sharpness · 401 papers
Keywords: sgd, sharpness, noise

Approaches in this cluster:

Most cited and most cited since 2024:

Implicit regularization · 190 papers
Keywords: implicit bias, regularization, bias gradient

Approaches in this cluster:

Most cited and most cited since 2024:

Related topics in Optimization

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy