AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Is SGD a Bayesian sampler? Well, almost

Chris Mingard, Guillermo Valle-Pérez, Joar Skalse, Ard A. Louis

JMLR 2021 · Optimization › Implicit bias and generalization · approach: Stability-based generalization analysis · 24 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase batch size from 32 to 128 (ablation)Adam, FCN on MNIST, CE loss · MNISTmean generalisation error (%): 1.13% -> 2.20%Hurt
increase batch size from 32 to 512 (ablation)Adam, FCN on MNIST, CE loss · MNISTmean generalisation error (%): 1.13% -> 2.67%Hurt
add 4x faster learning rate at batch size 512 (ablation)Adam, FCN on MNIST, CE loss · MNISTmean generalisation error (%): 2.67% -> 2.14%Improved
randomise 20% of training labels (ablation)FCN on MNIST, MSE loss · MNISTcorrelation between P_B and P_SGD: 13.4% and 5.80%Hurt
increase training set size from 1000 to 20000 (ablation)FCN on MNIST · MNISTbias towards low error functions and P_B/P_SGD correlation: |S|=1000 to |S|=20000Improved
add max-pooling to the CNN (ablation)CNN on Fashion-MNIST with Adam · Fashion-MNISTgeneralisation performanceImproved
use NTK instead of GP posteriorFCN on MNIST · MNISTfunction probabilitiesMixed
increase batch size under MSE loss (ablation)FCN on MNIST, MSE loss · MNISTgeneralisation errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy