Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use marginal-likelihood hyperparameter optimization instead of cross-validation | CNN · CIFAR10 | accuracy: 77.41 -> 80.46 | Helped |
| Use marginal-likelihood hyperparameter optimization instead of cross-validation | ResNet · CIFAR10 | accuracy: 83.73 -> 86.11 | Helped |
| Use marginal-likelihood hyperparameter optimization instead of cross-validation | MLP · FMNIST | accuracy: 88.09 -> 89.83 | Helped |
| Add damping to KFAC Laplace (ablation) | KFAC model on MNIST MLP · MNIST | accuracy: 98.38 -> 95.89 | Hurt |
| Add damping to KFAC Laplace (ablation) | KFAC model on MNIST MLP · MNIST | MargLik: -0.158 -> -2.199 | Hurt |
| Add damping to KFAC Laplace (ablation) | KFAC model on MNIST CNN · MNIST | accuracy: 99.46 -> 98.78 | Hurt |
| Add damping to KFAC Laplace (ablation) | KFAC model on FMNIST CNN · FMNIST | accuracy: 92.06 -> 89.54 | Hurt |
| Add damping to KFAC Laplace (ablation) | KFAC model on CIFAR10 CNN · CIFAR10 | accuracy: 80.46 -> 76.38 | Hurt |
| Use marginal-likelihood optimization rather than weak regularization | One-hidden-layer neural network · Synthetic banana classification | test accuracy: 86% -> 89% | Helped |
| Optimize architecture marginal likelihood after training | CNN versus MLP · FMNIST | rank correlation of test accuracy and marginal likelihood: 69% | Mixed |
| Increase ResNet width | ResNet · CIFAR-100 | marginal likelihood | Helped |
| Vary ResNet depth | ResNet · CIFAR-100 | marginal likelihood | No clear effect |
Closest papers by meaning
- Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels (ICML 2023 · similarity 0.91)
- Bayesian Model Selection, the Marginal Likelihood, and Generalization (ICML 2022 · similarity 0.91)
- Generative Marginalization Models (ICML 2024 · similarity 0.86)
- Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood (NeurIPS 2024 · similarity 0.85)
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.84)
- A Bayesian Approach to Data Point Selection (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.