Research map / Optimization / Implicit bias and generalization
A Bayesian Perspective on Training Speed and Model Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the sum of sequential posterior predictive log likelihoods as a marginal-likelihood estimator | Bayesian linear models · Feature dimension selection | agreement with marginal-likelihood model ranking | Helped |
| Increase the number of posterior samples used by the multi-sample estimator (ablation) | Marginal-likelihood estimator L_k · Feature dimension selection | bias relative to log marginal likelihood | Helped |
| Use exact posterior samples rather than gradient-descent-generated approximate posterior samples (ablation) | Marginal-likelihood estimator L_k · Feature dimension selection | gap from true marginal likelihood | Helped |
| Use concurrent predictive sampling rather than prior or posterior sampling (ablation) | Linear model combination · Linear model selection tasks | agreement of model ranking with log marginal likelihood | Helped |
| Use the convolutional NTK-GP rather than the fully connected MLP-NTK-GP | Infinite-width neural network Gaussian processes · MNIST regression | marginal-likelihood lower bound | Helped |
| Use minibatch sum of training losses as a predictor | Convolutional neural networks trained with SGD · FashionMNIST | association with average test cross-entropy | Helped |
| Train a linear ensemble that assigns weights based on model performance | Neural network architectures trained with SGD · FashionMNIST | ensemble weight for models with lower individual test error | Helped |
| Train the component models concurrently through the ensemble rather than individually in parallel (ablation) | Linear combination of neural networks · FashionMNIST | separation in assigned ensemble weights | Helped |
| Select subnetworks with lower sum of cross-entropy losses | Penultimate-layer subnetworks in a neural network · FashionMNIST | weight assigned by SGD | Helped |
| Select subnetworks with lower sum of cross-entropy losses | Penultimate-layer subnetworks in a neural network · CIFAR-10 | weight assigned by SGD | Helped |
Closest papers by meaning
- Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood (NeurIPS 2024 · similarity 0.87)
- Bayesian Low-Rank Learning (Bella): A Practical Approach to Bayesian Neural Networks (AAAI 2025 · similarity 0.86)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
- A Bayesian Approach to Data Point Selection (NeurIPS 2024 · similarity 0.86)
- Computation-Aware Gaussian Processes: Model Selection And Linear-Time Inference (NeurIPS 2024 · similarity 0.86)
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.