Research map / Learning theory and algorithms / Kernels and regression theory

Generalization Measures for Deep Learning Should Be Audited for Fragility

NeurIPS 2026 · Learning theory and algorithms › Kernels and regression theory · approach: Empirical generalization measures

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that post-mortem generalization measures for deep learning should be audited for fragility, since small routine training changes can shift their values, trends, or scaling behavior.

Problem

Post-mortem generalization measures computed on trained networks are used as explanations of why deep networks generalize, but they may be fragile to small, routine training changes that barely affect the predictor or test performance.

Approach

Proposes that generalization measures be audited for fragility (sensitivity to routine training changes that leave the learned predictor and test performance nearly unchanged) before being used as explanations of generalization.

Closest papers by meaning

• Exploring Generalization in Deep Learning (NeurIPS 2017 · similarity 0.89)
• Model-agnostic Measure of Generalization Difficulty (ICML 2023 · similarity 0.87)
• To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets (ICLR 2024 · similarity 0.86)
• Understanding Generalization through Decision Pattern Shift (NeurIPS 2026 · similarity 0.85)
• Grokking at the Edge of Numerical Stability (ICLR 2025 · similarity 0.85)
• Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.85)