Research map / Optimization / Implicit bias and generalization
Anticorrelated Noise Injection for Improved Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace independent perturbations with anticorrelated perturbation increments | Gradient descent on the widening valley · Widening valley | limiting expected squared norm ||u||²: Anti-PGD ≤ αD; PGD ≥ D/α | Helped |
| Replace independent perturbations with anticorrelated perturbation increments | Gradient descent · Quadratically-parametrized linear regression | test generalization | Helped |
| Replace independent perturbations with anticorrelated perturbation increments | Gradient descent · Matrix sensing | test generalization | Helped |
| Replace independent perturbations with anticorrelated perturbation increments | ResNet18-like model · CIFAR10 | test accuracy | Helped |
| Add anticorrelated noise after full-batch GD has converged to a sharp minimum (ablation) | ResNet18-like model · CIFAR10 | test accuracy | Helped |
| Add uncorrelated noise after full-batch GD has converged to a sharp minimum (ablation) | ResNet18-like model · CIFAR10 | test accuracy | Hurt |
| Combine minibatch noise with anticorrelated noise (ablation) | Anti-PGD · Appendix experiments | performance | Helped |
Closest papers by meaning
- On the Noisy Gradient Descent that Generalizes as SGD (ICML 2020 · similarity 0.90)
- Strength of Minibatch Noise in SGD (ICLR 2022 · similarity 0.88)
- On the Inherent Regularization Effects of Noise Injection During Training (ICML 2021 · similarity 0.87)
- Label Noise SGD Provably Prefers Flat Global Minimizers (NeurIPS 2021 · similarity 0.87)
- Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections (ICML 2021 · similarity 0.87)
- Estimating Generalization Performance Along the Trajectory of Proximal SGD in Robust Regression (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.