AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

How many degrees of freedom do we need to train deep networks: a loss landscape perspective

Brett W Larsen, Stanislav Fort, Nic Becker, Surya Ganguli

ICLR 2022 · Learning theory and algorithms › Kernels and regression theory · approach: High-dimensional asymptotic generalization · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Increase the training dimension of a random affine subspaceRandom affine subspace training · MNIST, Fashion MNIST, CIFAR-10, CIFAR-100, and SVHNsuccess probability of reaching a training loss sublevel setImproved
Require a lower target lossRandom and burn-in affine subspace training · MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100threshold training dimensionHurt
Increase burn-in steps before restricted training (ablation)Burn-in affine subspace training · MNIST, Fashion MNIST, CIFAR-10, and CIFAR-100threshold training dimensionImproved
Use burn-in initialization for Conv-3 on MNIST (ablation)Conv-3 affine subspace training · MNISTaccuracyNo clear effect
Train in a lottery subspace formed from principal components of the full training trajectoryLottery subspace training · MNIST, Fashion MNIST, CIFAR-10, CIFAR-100, and SVHNaccuracy at low training dimensionsImproved
Use lottery subspaces rather than random affine subspacesSubspace training · MNIST, Fashion MNIST, CIFAR-10, CIFAR-100, and SVHNtraining dimension needed to attain full accuracy: 20 to 60Improved
Use lottery tickets rather than random affine subspacesConv-2 on CIFAR-10 · CIFAR-10accuracyImproved
Use lottery tickets rather than random affine subspacesLottery ticket pruning · Experiments where methods overlapaccuracyNo clear effect
Increase lottery-ticket compression to high ratiosLottery ticket pruning · High-compression settingsaccuracyHurt
Replace the full network with its NTK linearization around a local optimum (ablation)Conv-2 · MNIST and Fashion MNISTthreshold training dimensionNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy