Research map / Optimization / Implicit bias and generalization
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use F-GR instead of B-GR | GR · CIFAR-10, 4-layer MLP | test accuracy: 58.3 ± 0.2 -> 58.6 ± 0.2 | Helped |
| Use F-GR instead of DB | GR · CIFAR-10, 4-layer MLP | test accuracy: 57.6 ± 0.2 -> 58.6 ± 0.2 | Helped |
| Use F-GR instead of B-GR | GR · CIFAR-10, ResNet-18 | test accuracy: 86.2 ± 0.3 -> 87.0 ± 0.2 | Helped |
| Use F-GR instead of DB | GR · CIFAR-10, ResNet-18 | test accuracy: 86.3 ± 0.3 -> 87.0 ± 0.2 | Helped |
| Use F-GR instead of B-GR | GR · CIFAR-10 w/o data augmentation, WRN-28-10 | test accuracy: 91.9 ± 0.1 -> 92.4 ± 0.1 | Helped |
| Use F-GR instead of DB | GR · CIFAR-10 w/o data augmentation, WRN-28-10 | test accuracy: 91.7 ± 0.2 -> 92.4 ± 0.1 | Helped |
| Use F-GR instead of B-GR | GR · CIFAR-10 with data augmentation, WRN-28-10 | test accuracy: 95.9 ± 0.1 -> 96.1 ± 0.1 | Helped |
| Use F-GR instead of DB | GR · CIFAR-10 with data augmentation, WRN-28-10 | test accuracy: 95.9 ± 0.1 -> 96.1 ± 0.1 | Helped |
| Use F-GR instead of B-GR | GR · CIFAR-100 w/o data augmentation, WRN-28-10 | test accuracy: 71.2 ± 0.6 -> 71.9 ± 0.1 | Helped |
| Use F-GR instead of DB | GR · CIFAR-100 w/o data augmentation, WRN-28-10 | test accuracy: 70.3 ± 0.2 -> 71.9 ± 0.1 | Helped |
| Use F-GR instead of B-GR | GR · CIFAR-100 with data augmentation, WRN-28-10 | test accuracy: 80.2 ± 0.2 -> 81.2 ± 0.2 | Helped |
| Use F-GR instead of DB | GR · CIFAR-100 with data augmentation, WRN-28-10 | test accuracy: 80.3 ± 0.4 -> 81.2 ± 0.2 | Helped |
Closest papers by meaning
- Implicit Gradient Regularization (ICLR 2021 · similarity 0.91)
- Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks (NeurIPS 2022 · similarity 0.90)
- When Will Gradient Regularization Be Harmful? (ICML 2024 · similarity 0.88)
- Estimating Implicit Regularization in Deep Learning (NeurIPS 2026 · similarity 0.88)
- Grokking at the Edge of Numerical Stability (ICLR 2025 · similarity 0.88)
- Improving Generalization and Convergence by Enhancing Implicit Regularization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.