AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Benign, Tempered, or Catastrophic: Toward a Refined Taxonomy of Overfitting

Neil Rohit Mallinar, James B Simon, Amirhesam Abedsoltan, Parthe Pandit, Misha Belkin, Preetum Nakkiran

NeurIPS 2022 · Learning theory and algorithms › Kernels and regression theory · approach: Double descent and overfitting · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add a nonzero ridge (delta = 0.1) to Gaussian-kernel regression (ablation)Gaussian kernel regression · Synthetic noise-only targets on sphere, d=5,10,15test MSE vs train samplesImproved
use Laplace kernel instead of Gaussian kernel, both ridgeless (ablation)Ridgeless kernel regression · Synthetic noise-only targets on spheretest MSE vs train samplesImproved
add a ridge to Laplace-kernel regression (ablation)Laplace kernel regression · Synthetic noise-only targets on spheretest MSEImproved
increase input dimension d for ridgeless Laplace kernel regressionLaplace kernel regression · Synthetic Laplace-kernel experiments (Appendix B)asymptotic excess MSEImproved
increase spectral decay exponent alpha of a powerlaw kernelRidgeless KR with powerlaw spectrum · Synthetic exact powerlaw spectraasymptotic test MSEHurt
increase label-flip noise in training setInterpolating Wide ResNet 28x10 · Binary CIFAR-10clean test errorHurt
increase train size n for interpolating ResNetsWide ResNet 28x10 · Binary CIFAR-10, n=5k to 40kclean test error under label noiseNo clear effect
train MLP for one epoch instead of to interpolation (ablation)MLP · Binary MNISTclean test error under label noiseImproved
use k-NN with k ~ log n instead of 1-NN (ablation)Nearest-neighbor classifier · Binary MNISTclean test error under label noiseImproved
add early stopping to Wide ResNet training (ablation)Wide ResNet · Binary SVHNclean test error under label noiseImproved
train interpolating three-layer MLP on a constant target with slight label noiseThree-layer MLP · Synthetic S9 inputs, f*(x)=1test errorHurt
continue training from early epochs to interpolationMLP on synthetic noisy data · Appendix E syntheticoverfitting regimeHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy