Research map / Optimization / Implicit bias and generalization
Noether’s Learning Dynamics: Role of Symmetry Breaking in Neural Networks
Hidenori Tanaka, Daniel Kunin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Allow filter norms to vary instead of constraining them to their initialization norm (ablation) | VGG11 with batch normalization · Tiny ImageNet, learning rate η ∼ 0.03 | test accuracy | Improved |
| Allow filter norms to vary instead of constraining them to their initialization norm (ablation) | VGG11 with batch normalization · Tiny ImageNet, learning rate η ∼ 0.001 | test accuracy | No clear effect |
| Use unconstrained rather than constrained filter norms (ablation) | VGG11 with batch normalization · Tiny ImageNet, large learning rate | loss | Improved |
| Add momentum (β ∼ 0.9) to amplify implicit adaptive optimization | VGG11 with batch normalization · Tiny ImageNet | test accuracy | Improved |
| Add weight decay | VGG11 with batch normalization · Tiny ImageNet | test accuracy | Improved |
Closest papers by meaning
- Symmetries in Overparametrized Neural Networks: A Mean Field View (NeurIPS 2024 · similarity 0.90)
- Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning (NeurIPS 2025 · similarity 0.88)
- Identifying Equivalent Training Dynamics (NeurIPS 2024 · similarity 0.88)
- Context-Informed Neural ODEs Unexpectedly Identify Broken Symmetries: Insights from the Poincaré–Hopf Theorem (ICML 2025 · similarity 0.88)
- A Riemannian Framework for Learning Reduced-order Lagrangian Dynamics (ICLR 2025 · similarity 0.87)
- Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
