AI Paper Atlas

Research map / Learning theory and algorithms / Kernels and regression theory

Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective

Keitaro Sakamoto, Issei Sato

NeurIPS 2022 · Learning theory and algorithms › Kernels and regression theory · approach: Statistical perspectives on generalization · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use SAM rather than SGD with a small learning rateIMP subnetworks · CIFAR10, VGG16, 90% and 95% sparsitytest accuracy: 91.8 -> 92.9; 91.8 -> 93.1Improved
Use SAM rather than SGD with a small learning rateIMP subnetworks · CIFAR10, ResNet20, 90% sparsitytest accuracy: 89.1 -> 89.7Improved
Use SAM rather than SGD with a small learning rateIMP subnetworks · CIFAR100, VGG16, 90% and 95% sparsitytest accuracy: 67.1 -> 70.4; 66.4 -> 70.5Improved
Use SAM rather than SGD with a small learning rateIMP subnetworks · CIFAR100, ResNet20, 95% sparsitytest accuracy: 45.2 -> 45.2No clear effect
Use NVRM rather than SGD with a small learning rateIMP subnetworks · CIFAR100, VGG16, 90% and 95% sparsitytest accuracy: 67.1 -> 71.2; 66.4 -> 71.0Improved
Use NVRM rather than SGD with a small learning rateIMP subnetworks · CIFAR10, ResNet20test accuracy: 89.1 -> 89.3; 87.3 -> 87.1No clear effect
Use a small learning rate rather than a large learning rateIMP · CIFAR10 and CIFAR100 label-noise experimentstest accuracyMixed
Add sharpness-aware minimization to small-learning-rate IMPIMP subnetworks · CIFAR10 and CIFAR100test accuracyImproved
Add sharpness-aware minimization to large-learning-rate IMP (ablation)IMP subnetworks · CIFAR10, ResNet20test accuracyNo clear effect
Add regularization toward the initial weights (ablation)Large-learning-rate IMP · CIFAR10, ResNet20test accuracyImproved
Add L2 norm regularization instead of regularizing distance from initialization (ablation)Small-learning-rate IMP · CIFAR10, ResNet20winning-ticket test accuracyHurt
Use a spike-and-slab PAC-Bayes distribution with initialization as prior mean instead of a zero-mean Gaussian (ablation)PAC-Bayes analysis of IMP tickets · CIFAR10, ResNet20, 90% sparsityalignment with observed test accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy