Research map / Learning theory and algorithms / Prediction and decision losses
Large Scale Structure of Neural Network Loss Landscapes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase learning rate | Neural network training · CIFAR-10 | radial low-loss tunnel width | Helped |
| Increase batch size | Neural network training · Neural network loss landscapes | radial low-loss tunnel width | Hurt |
| Increase L2 regularization | Neural network training · CIFAR-10 | radial low-loss tunnel width | Mixed |
| Increase dropout rate | Neural network training · CIFAR-10 | radial low-loss tunnel width | Mixed |
| Use a high maximum learning rate during cyclical snapshotting and average snapshot weights | Stochastic Weight Averaging · CIFAR-10 | accuracy | Hurt |
| Use a low learning rate so snapshots remain within the same wedge before averaging their weights | Stochastic Weight Averaging · CIFAR-10 | accuracy | Helped |
| Construct low-loss connectors between multiple optima, rather than only connecting pairs | Neural network loss landscapes · CIFAR-10 and Fashion MNIST | low-loss connectivity | Helped |
Closest papers by meaning
- Loss Landscape Characterization of Neural Networks without Over-Parametrization (NeurIPS 2024 · similarity 0.91)
- Super Consistency of Neural Network Landscapes and Learning Rate Transfer (NeurIPS 2024 · similarity 0.89)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.89)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.89)
- Input Space Mode Connectivity in Deep Neural Networks (ICLR 2025 · similarity 0.88)
- Flat Channels to Infinity in Neural Loss Landscapes (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.