Research map / Learning theory and algorithms / Kernels and regression theory
Deep Learning meets Nonparametric Regression: Are Weight-Decayed DNNs Locally Adaptive?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use deeper parallel ReLU networks | Parallel NN with ℓ2 regularization · Besov and BV classes | MSE rate: p = 2/L; e^{-c_6 L} | Helped |
| Use depth C log n ≤ L ≤ 100 C log n | Parallel ReLU NN with ℓ2 regularization · Besov class | MSE rate: \tilde{O}(n^{-\frac{2\alpha}{2\alpha +d}(1-o(1))} ) | Helped |
| Tune only the regularization factor rather than the architecture for each smoothness class | Parallel ReLU NN · Besov classes with varying smoothness | MSE rate | Helped |
| Use parallel NN instead of linear estimators | Regression on BV(m), d=1 · BV(m) | MSE rate: O(n^{-(2m+2)/(2m+3)}); O(n^{-(2m+1)/(2m+2)}) | Helped |
| Use Parallel NN instead of smoothing splines | Doppler and Vary regression · Doppler and Vary | MSE | Helped |
| Use Parallel NN instead of trend filtering | Doppler and Vary regression · Doppler and Vary | MSE | Mixed |
| Use Parallel NN instead of smoothing splines | Regression on Vary · Vary | MSE rate: n^{-4/5}; n^{-3/4} | Helped |
| Use vanilla NN instead of a linear-rate method | Regression on Vary · Vary | MSE rate: n^{-4/5} | Helped |
Closest papers by meaning
- Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes (NeurIPS 2024 · similarity 0.88)
- On the Hardness of Training Deep Neural Networks Discretely (AAAI 2025 · similarity 0.87)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.87)
- Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes (ICML 2024 · similarity 0.87)
- Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time (ICML 2024 · similarity 0.87)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.