Research map / Learning theory and algorithms / Kernels and regression theory
Bayes-optimal Learning of Deep Random Networks of Extensive-width
Hugo Cui, Florent Krzakala, Lenka Zdeborova
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use optimally regularized ridge regression instead of other estimators | Deep extensive-width random network regression · Proportional regime | mean squared error | No clear effect |
| Use optimally regularized kernel regression instead of other estimators | Deep extensive-width random network regression · Proportional regime | mean squared error | No clear effect |
| Use random features regression rather than the original input-space linear estimator | Deep extensive-width random network regression · Proportional regime | mean squared error | Hurt |
| Use logistic regression rather than Bayes-optimal inference | Deep extensive-width random network classification · Proportional regime | classification error: 10^{-4} | Hurt |
| Use ridge classification rather than Bayes-optimal inference | Deep extensive-width random network classification · Proportional regime | classification error: 10^{-3} | Hurt |
| Train a neural network end-to-end instead of using kernel regression | ReLU target · Quadratic sample regime | mean squared error: 10^{-5}; 10^3 | Improved |
| Train a neural network end-to-end instead of using ridge regression | ReLU target · Quadratic sample regime | mean squared error: 10^{-5} | Improved |
| Apply the closed-form covariance recursion | Deep random networks · Activation covariance, tanh | relative squared Frobenius distance: 0.005 | Improved |
| Apply the closed-form covariance recursion | Deep random networks · Activation covariance, sign | relative squared Frobenius distance: 0.008 | Improved |
| Apply the closed-form covariance recursion | Deep random networks · Activation covariance, erf | relative squared Frobenius distance: 0.004 | Improved |
Closest papers by meaning
- Bayes-optimal learning of an extensive-width neural network from quadratically many samples (NeurIPS 2024 · similarity 0.93)
- Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization (NeurIPS 2020 · similarity 0.89)
- Deep Neural Networks as Gaussian Processes (ICLR 2018 · similarity 0.88)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.87)
- Critical feature learning in deep neural networks (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
