Research map / Optimization / Optimizers
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| subset-of-data KFAC bound (batch 500, single output) instead of full-batch KFAC, prior precision | layer-wise prior precision on Wide ResNet 16-4 · CIFAR-10 | accuracy [%]: 88.2 vs 89.2 accuracy, at 65% of runtime | No clear effect |
| subset-of-data NTK/KFAC bounds (batch 150, single output) instead of full-batch KFAC for invariance learning | affine invariance learning on Wide ResNet 16-4 · CIFAR-10 | accuracy [%]: KFAC-150-1 92.5 and NTK-150-1 90.3 vs full 93.2; runtime 10% and 9% | No clear effect |
| single-output (class-wise) partitioning for full-dataset KFAC with random output sampling (ablation) | layer-wise prior precision on Wide ResNet 16-4 · CIFAR-100 | accuracy [%]: 66.8 vs 64.6 for full KFAC-N-C | Helped |
| subset-of-data KFAC bound with 10 outputs (batch 500) vs single output | prior precision on Wide ResNet 16-4 · CIFAR-100 | accuracy [%]: 67.9 vs 66.8 (KFAC-500-10 vs KFAC-N-1) | Helped |
| subset-of-data KFAC bound for invariance learning (batch 150, single output) vs full-data KFAC bound | invariance learning on Wide ResNet 16-4 · CIFAR-100 | accuracy [%]: 71.8 vs 71.7 for KFAC-N-1 at 5% vs 10% time | No clear effect |
| subset-of-data KFAC bound (60 points, single output) for invariance learning vs Augerino/partitioned baselines | ResNet-50 on TinyImageNet · TinyImageNet | accuracy [%]: 58.4 vs 48.6 (partitioned) and 41.1 (Augerino) | Helped |
| subset-of-data KFAC bound (400 points) for prior precision learning vs default baseline | ResNet-50 on TinyImageNet · TinyImageNet | accuracy [%]: 53.0 vs 45.6 baseline | Helped |
| subset-of-data NTK bound (400 points) instead of KFAC subset bound for prior precision (ablation) | ResNet-50 on TinyImageNet · TinyImageNet | accuracy [%]: 49.8 vs 53.0 for KFAC-400-1 | Hurt |
| subset-of-data NTK bound (70 points, single output) vs KFAC (60 points, single output) for invariance learning (ablation) | ResNet-50 on TinyImageNet · TinyImageNet | accuracy [%]: 56.3 for NTK-70-1 vs 58.4 for KFAC-60-1 | Hurt |
| output-wise (single-output) partitioning of NTK/KFAC bound vs full-output bound (ablation) | prior precision learning on MNIST subset (small CNN) · MNIST subset (1000 digits) | bound tightness: almost as tight as full, at more than C times smaller cost | No clear effect |
| grouping mini-batches by class label instead of random partition (ablation) | prior precision learning · CIFAR/MNIST prior precision | bound and test log likelihood: about 10% improvement on average for prior precision; no significant difference for invariance learning | Helped |
Closest papers by meaning
- Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning (ICML 2021 · similarity 0.91)
- FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learning (NeurIPS 2024 · similarity 0.88)
- Variational Linearized Laplace Approximation for Bayesian Deep Learning (ICML 2024 · similarity 0.86)
- Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation (NeurIPS 2024 · similarity 0.85)
- Kernel-based guarantees for nonlinear parametric models in Bayesian optimization (NeurIPS 2026 · similarity 0.85)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.