Research map / Learning theory and algorithms / Kernels and regression theory
Why bigger is not always better: on finite and infinite neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a finite-width network rather than its infinite-width limit | Bayesian neural network · Toy two-layer linear-network simulations with model mismatch | predictive log-probability | Helped |
| Increase hidden-layer width toward the infinite-width limit (ablation) | Bayesian neural network · Toy two-layer linear-network simulations with model mismatch | predictive log-probability | Hurt |
| Increase network depth (ablation) | Finite fully connected linear network · Prior kernel-flexibility analysis | kernel variance | Helped |
| Increase network width (ablation) | Finite fully connected linear network · Prior kernel-flexibility analysis | kernel variance | Hurt |
| Use convolutional rather than locally connected layers with spatially structured inputs (ablation) | Finite neural network · Prior kernel-flexibility analysis | kernel variance | Helped |
| Use locally connected layers rather than convolutional layers (ablation) | Finite neural network · Prior kernel-flexibility analysis with spatially structured inputs | kernel variance | Hurt |
| Train the finite ResNet with SGD | 34-layer ResNet · CIFAR-10 test data | correlation with the infinite-network kernel | Hurt |
| Train the finite ResNet with SGD | 34-layer ResNet · CIFAR-10 test data | correlation with the output kernel | Helped |
| Increase the number of intermediate network units relative to output units (ablation) | Bayesian deep linear network with posterior sampling · Posterior kernel analysis | representation learning toward the output kernel | Hurt |
| Use MAP inference instead of posterior sampling (ablation) | Finite deep linear network · Posterior kernel analysis | representation shaping across network widths | Mixed |
Closest papers by meaning
- An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network (ICML 2024 · similarity 0.87)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.87)
- Partially Stochastic Infinitely Deep Bayesian Neural Networks (ICML 2024 · similarity 0.87)
- Algorithm Development in Neural Networks: Insights from the Streaming Parity Task (ICML 2025 · similarity 0.86)
- Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks (NeurIPS 2026 · similarity 0.86)
- Deep Kernel Posterior Learning under Infinite Variance Prior Weights (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.