Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Dissecting Non-Vacuous Generalization Bounds based on the Mean-Field Approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Center the isotropic Gaussian prior at random initialization rather than zero (ablation) | PAC-Bayes bounds with diagonal Gaussian prior and posterior · MNIST | bound quality | Helped |
| Optimize diagonal posterior covariance with mean-field VI rather than use an isotropic posterior (ablation) | PAC-Bayes bounds with prior centered at random initialization · MNIST | bound quality | No clear effect |
| Use mean-field VI rather than an isotropic posterior (ablation) | PAC-Bayes bounds · CIFAR | bound quality | No clear effect |
| Optimize posterior covariance in closed form rather than with VI (ablation) | PAC-Bayes bounds with a quadratic loss approximation · MNIST | bound quality: between 5-10% | Helped |
| Optimize prior and posterior covariances jointly rather than optimize only the posterior covariance (ablation) | Mean-field approximation sanity-check bounds · MNIST | bound quality: 5-10% | Helped |
| Use block-diagonal covariance rather than mean-field covariance (ablation) | PAC-Bayes bounds · MNIST10 | estimated complexity: 8.2% | Helped |
| Use block-diagonal covariance rather than mean-field covariance (ablation) | PAC-Bayes bounds · MNIST5 | estimated complexity: 7.5% | Helped |
| Use block-diagonal covariance rather than mean-field covariance (ablation) | PAC-Bayes bounds · MNIST2 | estimated complexity: 4.4% | Helped |
| Use block-diagonal posterior covariance rather than mean-field covariance (ablation) | PAC-Bayes bounds · MNIST10 and MNIST5 | estimated complexity: ~10% | Helped |
| Change prior centering or optimize posterior covariance (ablation) | PAC-Bayes bounds with diagonal Gaussian prior and posterior · CIFAR | bound quality | No clear effect |
| Use optimized diagonal posterior and prior covariances (ablation) | PAC-Bayes bounds with a quadratic loss approximation · CIFAR | bound quality | No clear effect |
Closest papers by meaning
- On the Importance of Gradient Norm in PAC-Bayesian Bounds (NeurIPS 2022 · similarity 0.90)
- Collapsed Variational Bounds for Bayesian Neural Networks (NeurIPS 2021 · similarity 0.90)
- Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience (ICLR 2019 · similarity 0.89)
- PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization (NeurIPS 2022 · similarity 0.88)
- Non-Vacuous Generalisation Bounds for Shallow Neural Networks (ICML 2022 · similarity 0.88)
- Certifying Deep Network Risks and Individual Predictions with PAC-Bayes Loss via Localized Priors (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.