Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Deep Neural Networks as Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use exact Bayesian inference with the infinite-width neural-network GP instead of a finite-width NN trained with Adam | Fully connected ReLU networks · MNIST:1k, MNIST:50k, CIFAR-10:1k, CIFAR-10:10k, CIFAR-10:45k | test accuracy: 0.9252 -> 0.9279; 0.9864 -> 0.9875; 0.3225 -> 0.3608; 0.4545 -> 0.4780; 0.5313 -> 0.5566 | Helped |
| Use exact Bayesian inference with the infinite-width neural-network GP instead of a finite-width NN trained with Adam | Fully connected ReLU networks · MNIST:10k | test accuracy: 0.9771 -> 0.9765 | Hurt |
| Use exact Bayesian inference with the infinite-width neural-network GP instead of a finite-width NN trained with Adam | Fully connected Tanh networks · MNIST:1k, MNIST:10k, MNIST:50k, CIFAR-10:1k, CIFAR-10:10k, CIFAR-10:45k | test accuracy: 0.9254 -> 0.9266; 0.9745 -> 0.9773; 0.9857 -> 0.9879; 0.3378 -> 0.3702; 0.4429 -> 0.4766; 0.5034 -> 0.5558 | Helped |
| Increase finite-network layer width | Fully connected NNs · MNIST and CIFAR-10 | test performance | Helped |
| Set NNGP weight and bias variances near the signal-propagation critical line | NNGP · MNIST and CIFAR-10 | test accuracy | Helped |
| Increase depth far from the critical line | NNGP · MNIST and CIFAR-10 | test accuracy | Hurt |
Closest papers by meaning
- Approximate Inference Turns Deep Networks into Gaussian Processes (NeurIPS 2019 · similarity 0.91)
- Deep Kernel Posterior Learning under Infinite Variance Prior Weights (ICLR 2025 · similarity 0.90)
- Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes (ICLR 2019 · similarity 0.90)
- Critical feature learning in deep neural networks (ICML 2024 · similarity 0.88)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.88)
- An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.