Research map / Learning theory and algorithms / Kernels and regression theory
Random deep neural networks are biased towards simple functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a random wide deep ReLU network prior instead of a uniformly random binary classifier | Binary classification of bit strings · Bit strings of length n | average nearest differently classified Hamming distance: one; at least $\sqrt{n/(2\pi\ln n)}$ | Helped |
| Use a random wide deep ReLU network prior instead of a uniformly random binary classifier | Binary classification of bit strings · Bit strings of length n under random bit flips | average flips to change classification: two; at least $n/4$ | Helped |
| Train the network on MNIST even/odd examples | Two-layer fully-connected network · MNIST training and test sets | nearest-boundary Hamming distance: $1.42$ and $10.61$ for incorrectly and correctly classified training pictures; $2.30$ and $10.47$ for test pictures | Mixed |
| Use training or test images rather than randomly sampled bit strings | Trained MNIST networks · MNIST versus random binary inputs | nearest-boundary Hamming distance: random bits: approximately 4.33 | Helped |
Closest papers by meaning
- Deep learning generalizes because the parameter-function map is biased towards simple functions (ICLR 2019 · similarity 0.89)
- Neural Redshift: Random Networks are not Random Functions (CVPR 2024 · similarity 0.86)
- Simplicity Bias in Overparameterized Machine Learning (AAAI 2024 · similarity 0.86)
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.85)
- How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers (ICML 2024 · similarity 0.85)
- Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.