Research map / Learning theory and algorithms / Kernels and regression theory
Sensitivity and Generalization in Neural Networks: an Empirical Study
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use correct labels instead of random labels | fully connected neural networks · CIFAR10 | generalization gap | Helped |
| add data augmentation | fully connected neural networks · CIFAR10 | test accuracy | Helped |
| use ReLU instead of HardSigmoid | fully connected neural networks · CIFAR10 | Jacobian norm | Helped |
| use mini-batch optimization instead of full-batch training | fully connected neural networks · CIFAR10 | generalization gap | Helped |
| evaluate sensitivity near training data rather than along a random ellipse away from the data | trained MNIST network · MNIST | Jacobian norm | Helped |
| use a trajectory through same-class training points rather than different-class points | trained MNIST network · MNIST | Jacobian norm | Helped |
| compare networks using transition count instead of Jacobian norm | fully connected neural networks · CIFAR10 and CIFAR100 | correlation with generalization gap | Hurt |
| use Jacobian norm as a sensitivity measure across varied architectures and optimization choices | fully connected neural networks · CIFAR10 and CIFAR100 | correlation with generalization gap | Helped |
| use Jacobian norm to predict individual-point loss | trained neural networks · CIFAR10 | cross-entropy loss | Mixed |
| use Jacobian norm to predict individual-point loss | networks trained with ℓ2-loss · CIFAR10 | ℓ2-loss | Mixed |
Closest papers by meaning
- Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks (ICML 2019 · similarity 0.90)
- The role of over-parametrization in generalization of neural networks (ICLR 2019 · similarity 0.90)
- How much does Initialization Affect Generalization? (ICML 2023 · similarity 0.90)
- Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study (NeurIPS 2025 · similarity 0.89)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.89)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.