Research map / Learning theory and algorithms / Kernels and regression theory
Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks Using PAC-Bayesian Analysis
Yusuke Tsuzuku, Issei Sato, Masashi Sugiyama
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use normalized sharpness instead of trace of the Hessian | Wide ResNet · CIFAR-10, varying random-label ratios | correlation coefficient with generalization gap: 70.8 -> 91.1 | Improved |
| Use normalized sharpness to distinguish models trained on random labels | MLP and LeNet · MNIST | association between sharpness and generalization gap | Improved |
| Use normalized sharpness to distinguish models trained on random labels | LeNet and Wide ResNet · CIFAR-10 | association between sharpness and generalization gap | Improved |
| Remove scale normalization from the flatness measure | Flatness measures on tested networks · MNIST and CIFAR-10 | ability to distinguish random-label models | Hurt |
| Use an alternative direct optimization definition of normalized sharpness instead of the second-order approximation | Normalized sharpness calculation · Not evaluated experimentally | optimization convexity | Hurt |
| Use an alternative direct optimization definition of normalized sharpness instead of the second-order approximation | Normalized sharpness calculation · Not evaluated experimentally | sensitivity to the choice of lambda | Hurt |
Closest papers by meaning
- Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel (ICLR 2023 · similarity 0.92)
- Sharp Minima Can Generalize For Deep Nets (ICML 2017 · similarity 0.90)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.89)
- Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization (CVPR 2023 · similarity 0.89)
- Entropic gradient descent algorithms and wide flat minima (ICLR 2021 · similarity 0.89)
- Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization (NeurIPS 2023 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
