Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian Model Selection, the Marginal Likelihood, and Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace LML with CLML for model comparison | Fourier model selection · Fourier model | model preference aligned with generalization: LML: 53.8 (M3) vs 28.9 (M9); CLML: 28.9 (M9) vs 11.45 (M3) | Helped |
| Use a diffuse prior with the Laplace approximation | Bayesian neural network prior-scale estimation · Sinusoidal likelihood toy example | marginal likelihood estimate accuracy | Hurt |
| Use CLML instead of LML for neural architecture ranking | CNN and ResNet architecture comparison · CIFAR-10 and CIFAR-100 | correlation with generalization | Helped |
| Optimize prior precision before ranking architectures | Laplace LML architecture selection · CIFAR-10 and CIFAR-100 | correlation with BMA test accuracy | Helped |
| Optimize layer-wise rather than global prior precision (ablation) | Laplace LML architecture selection · CIFAR-10 and CIFAR-100 | correlation with BMA test accuracy | Helped |
| Calibrate models with temperature scaling (ablation) | CLML-based CNN ranking · CIFAR-10 CNNs | correlation with BMA test accuracy | Helped |
| Replace LML optimization with CLML optimization | Deep kernel learning · UCI regression tasks, limited training data | test performance | Helped |
| Replace LML optimization with CLML optimization | Deep Kernel Transfer · Omniglot to EMNIST | classification error | Helped |
| Replace LML optimization with CLML optimization | Deep Kernel Transfer · QMUL head pose dataset shift | MSE | Helped |
| Use LML rather than test predictive performance to select prior variance | Gaussian-process density model · Gaussian mean estimation | alignment with test predictive log-likelihood | Hurt |
| Use LML rather than CLML to select the RQ-kernel α hyperparameter under overestimated observation noise | Gaussian process with rational quadratic kernel · RQ-kernel hyperparameter selection | alignment with test log-likelihood | Hurt |
| Increase dataset size when comparing LML- and CLML-tuned DKL (ablation) | Deep kernel learning · UCI regression tasks | performance gap between CLML and LML | No clear effect |
Closest papers by meaning
- A Bayesian Perspective on Training Speed and Model Selection (NeurIPS 2020 · similarity 0.91)
- Generative Marginalization Models (ICML 2024 · similarity 0.88)
- Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood (NeurIPS 2024 · similarity 0.87)
- Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss (NeurIPS 2024 · similarity 0.87)
- Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference (ICML 2024 · similarity 0.87)
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.