Research map / Learning theory and algorithms / Prediction and decision losses
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reparametrize the network with WeightNorm and compute the Laplace LML naively | Neural network · Laplace LML comparison | Laplace log-marginal likelihood: -212.7$\pm$3.4 -> -227.1$\pm$3.6 | Hurt |
| Reparametrize the network with WeightNorm and compute the loss term | Neural network · Laplace LML comparison | -L: -143.4$\pm$0.0 -> -143.4$\pm$0.0 | No clear effect |
| Reparametrize the network with WeightNorm and compute the remaining Laplace terms naively | Neural network · Laplace LML comparison | Rest: -69.3$\pm$3.4 -> -83.7$\pm$3.6 | Hurt |
| Track the metric and transform it with the Hessian when forming the Hessian operator | Neural networks under reparametrization · Hessian-based flatness measures at minima | Hessian-operator determinant, trace, and eigenvalues | Helped |
| Transform the metric along with gradients under reparametrization | Gradient descent and preconditioned gradient descent · Reparametrized parameter spaces | ODE trajectory invariance | Helped |
| Express and transform probability densities relative to the Riemannian volume form | Probability-density maximization · Reparametrized parameter spaces | Density-mode invariance | Helped |
| Treat NTP as a mere reparametrization of SP | Standard and NTK parameterizations · Infinite-width neural networks | Limiting-kernel agreement | Hurt |
Closest papers by meaning
- Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture (ICLR 2025 · similarity 0.88)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.87)
- Global curvature for second-order optimization of neural networks (ICML 2025 · similarity 0.87)
- Parameter symmetries determine representational geometry in overparameterized nonlinear networks (NeurIPS 2026 · similarity 0.86)
- Reparameterization invariance in approximate Bayesian inference (NeurIPS 2024 · similarity 0.86)
- Geometry-Aware Similarity Metrics for Neural Representations on Riemannian and Statistical Manifolds (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.