Research map / Optimization / Implicit bias and generalization
Topology and Geometry of Half-Rectified Network Optimization
C. Daniel Freeman, Joan Bruna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a half-rectified nonlinear network instead of a linear network | Half-rectified networks · Adversarially constructed data distributions | level-set connectedness | Hurt |
| Apply ridge regularization to a two-layer linear network | Two-layer linear network · Empirical and oracle risk | level-set connectedness | Improved |
| Use the Dynamic String Sampling algorithm to estimate paths between trained models | CNNs on MNIST and CIFAR-10; LSTM on PTB · MNIST, CIFAR-10, PTB next-word prediction | level-set connectivity | Improved |
| Train models to lower loss or perplexity | Models evaluated with Dynamic String Sampling · Polynomial regression, MNIST, CIFAR-10, PTB | normalized geodesic length | Hurt |
| Train models to lower loss or perplexity | Models evaluated with Dynamic String Sampling · Polynomial regression, MNIST, CIFAR-10, PTB | number of intermediate beads | Hurt |
Closest papers by meaning
- Directional convergence and alignment in deep learning (NeurIPS 2020 · similarity 0.89)
- The Loss Surface Of Deep Linear Networks Viewed Through The Algebraic Geometry Lens (TPAMI 2021 · similarity 0.89)
- No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths (ICML 2024 · similarity 0.89)
- Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms (NeurIPS 2024 · similarity 0.88)
- Flat Channels to Infinity in Neural Loss Landscapes (NeurIPS 2025 · similarity 0.88)
- Exploring The Loss Landscape Of Regularized Neural Networks Via Convex Duality (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
