Research map / Learning theory and algorithms / Kernels and regression theory
On the Inherent Regularization Effects of Noise Injection During Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Inject Gaussian noise into training inputs | Random feature model · Generalization error at complexity η lower than ℓ | generalization error | Helped |
| Use ReLU or binary-step activation with noise injection | Noisy formulation · Nonlinear regression; generalization error across complexity η | generalization error | Helped |
| Use SoftPlus activation with noise injection | Noisy formulation · Nonlinear regression; generalization error across complexity η | generalization error | Mixed |
| Increase the input noise variance | Noisy formulation · Binary classification, especially at low η | generalization error | Helped |
| Select an appropriate noise variance | Noisy formulation · Binary classification; double-descent regime | generalization error | Helped |
| Replace input noise injection with dropout | Noisy formulation · Binary classification with sigmoid activation | generalization error | No clear effect |
| Replace input noise injection with dropout | Noisy formulation · Binary classification with Erf activation | generalization error | Helped |
| Replace input noise injection with dropout | Noisy formulation · Binary classification with ReLU activation | generalization error | Helped |
| Replace input noise injection with dropout | Noisy formulation · Binary classification with sign activation at low η | generalization error | Hurt |
| Use dropout instead of input noise injection | Noisy formulation · Generalization error as the number of perturbations grows | convergence rate | Helped |
| Increase the number of input noise samples | Noisy formulation · Generalization error as ℓ grows | generalization error | No clear effect |
Closest papers by meaning
- Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization (NeurIPS 2017 · similarity 0.90)
- Dropout Training, Data-dependent Regularization, and Generalization Bounds (ICML 2018 · similarity 0.89)
- Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation (NeurIPS 2026 · similarity 0.88)
- Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks (ICLR 2024 · similarity 0.87)
- Ghost Noise for Regularizing Deep Neural Networks (AAAI 2024 · similarity 0.86)
- How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.