Research map / Learning theory and algorithms / Kernels and regression theory
Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective
Keitaro Sakamoto, Issei Sato
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAM rather than SGD with a small learning rate | IMP subnetworks · CIFAR10, VGG16, 90% and 95% sparsity | test accuracy: 91.8 -> 92.9; 91.8 -> 93.1 | Improved |
| Use SAM rather than SGD with a small learning rate | IMP subnetworks · CIFAR10, ResNet20, 90% sparsity | test accuracy: 89.1 -> 89.7 | Improved |
| Use SAM rather than SGD with a small learning rate | IMP subnetworks · CIFAR100, VGG16, 90% and 95% sparsity | test accuracy: 67.1 -> 70.4; 66.4 -> 70.5 | Improved |
| Use SAM rather than SGD with a small learning rate | IMP subnetworks · CIFAR100, ResNet20, 95% sparsity | test accuracy: 45.2 -> 45.2 | No clear effect |
| Use NVRM rather than SGD with a small learning rate | IMP subnetworks · CIFAR100, VGG16, 90% and 95% sparsity | test accuracy: 67.1 -> 71.2; 66.4 -> 71.0 | Improved |
| Use NVRM rather than SGD with a small learning rate | IMP subnetworks · CIFAR10, ResNet20 | test accuracy: 89.1 -> 89.3; 87.3 -> 87.1 | No clear effect |
| Use a small learning rate rather than a large learning rate | IMP · CIFAR10 and CIFAR100 label-noise experiments | test accuracy | Mixed |
| Add sharpness-aware minimization to small-learning-rate IMP | IMP subnetworks · CIFAR10 and CIFAR100 | test accuracy | Improved |
| Add sharpness-aware minimization to large-learning-rate IMP (ablation) | IMP subnetworks · CIFAR10, ResNet20 | test accuracy | No clear effect |
| Add regularization toward the initial weights (ablation) | Large-learning-rate IMP · CIFAR10, ResNet20 | test accuracy | Improved |
| Add L2 norm regularization instead of regularizing distance from initialization (ablation) | Small-learning-rate IMP · CIFAR10, ResNet20 | winning-ticket test accuracy | Hurt |
| Use a spike-and-slab PAC-Bayes distribution with initialization as prior mean instead of a zero-mean Gaussian (ablation) | PAC-Bayes analysis of IMP tickets · CIFAR10, ResNet20, 90% sparsity | alignment with observed test accuracy | Improved |
Closest papers by meaning
- Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks (NeurIPS 2021 · similarity 0.92)
- Dual Lottery Ticket Hypothesis (ICLR 2022 · similarity 0.91)
- Find A Winning Sign: Sign Is All We Need to Win the Lottery (ICLR 2025 · similarity 0.91)
- Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets (ICML 2022 · similarity 0.91)
- Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Win (ICML 2025 · similarity 0.88)
- On the Sparsity of the Strong Lottery Ticket Hypothesis (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
