Research map / Learning theory and algorithms / Prediction and decision losses
Why neural networks find simple solutions: The many regularizers of geometric complexity
Benoit Dherin, Michael Munn, Mihaela Rosca, David GT Barrett
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase L2, spectral or flatness regularization strength | ResNet18/MLP · CIFAR10/MNIST | geometric complexity | Improved |
| increase label noise proportion | ResNet18, MLP · CIFAR10, MNIST | geometric complexity | Improved |
| use larger learning rate | ResNet18 · CIFAR10 | geometric complexity (lower) and test accuracy | Improved |
| use smaller batch size | ResNet18 · CIFAR10 | geometric complexity (lower) and test accuracy | Improved |
| add explicit GC (Jacobian) regularization | MLP, ResNet18 · MNIST, CIFAR10 | test accuracy | Improved |
| increase network depth at initialization | ReLU MLP · initialization | geometric complexity | Improved |
Closest papers by meaning
- Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms (NeurIPS 2024 · similarity 0.88)
- The Impact of Geometric Complexity on Neural Collapse in Transfer Learning (NeurIPS 2024 · similarity 0.88)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.87)
- Sample complexity of data-driven tuning of model hyperparameters in neural networks with structured parameter-dependent dual function (NeurIPS 2025 · similarity 0.87)
- Exploring the Complexity of Deep Neural Networks through Functional Equivalence (ICML 2024 · similarity 0.85)
- Complexity Scaling Laws for Neural Models using Combinatorial Optimization (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
