Research map / Learning theory and algorithms / Kernels and regression theory
Relative Flatness and Generalization
Henning Petzka, Michael Kamp, Linara Adilova, Cristian Sminchisescu, Mario Boley
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase class separation (more locally constant labels) in synthetic data (ablation) | relative flatness measure · Synthetic binary classification | correlation with generalization gap | Improved |
| use relative flatness instead of trace of Hessian, Fisher-Rao norm, PAC-Bayes measure or weight norm | LeNet5 local minima · CIFAR10, 110 local minima | correlation with generalization gap | Improved |
| apply layer-wise reparameterization to trained networks (ablation) | relative flatness and Fisher-Rao norm · CIFAR10, LeNet5 | correlation with generalization gap | No clear effect |
| apply layer-wise reparameterization to trained networks (ablation) | trace of Hessian, PAC-Bayes measure, weight norm · CIFAR10, LeNet5 | correlation with generalization gap | Hurt |
| compute relative flatness on a fully-connected layer other than the penultimate layer (ablation) | relative flatness · CIFAR10, LeNet5 | correlation with generalization gap | Mixed |
| add KDE-based approximation of representativeness to relative flatness | generalization bound · Synthetic data, 3-fold CV | bound tightness on generalization gap | Improved |
| make the learning problem easier (larger class separation) | approximated generalization bound · Synthetic data | approximated bound value | Improved |
Closest papers by meaning
- Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking (NeurIPS 2025 · similarity 0.89)
- Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization (NeurIPS 2023 · similarity 0.88)
- Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks Using PAC-Bayesian Analysis (ICML 2020 · similarity 0.88)
- Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning (ICLR 2024 · similarity 0.86)
- Simplicity Bias via Global Convergence of Sharpness Minimization (ICML 2024 · similarity 0.86)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
