Research map / Label-efficient and robust learning / Recognition and long-tail
Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use inconsistency instead of 1-sharpness to predict generalization gap | gap predictor · ImageNet ResNet50 (Case 1) | prediction error: 0.21 -> 0.13 | Helped |
| use inconsistency instead of Hessian to predict generalization gap | gap predictor · ImageNet ViT (Case 2) | prediction error: 1.00 -> 0.10 | Helped |
| use inconsistency instead of 1-sharpness | gap predictor · Car fine-tuning (Case 7) | prediction error: 0.19 vs 0.24 inconsistency; dev 1-sharpness 0.14 | Hurt |
| use inconsistency instead of Hessian of dev. loss | gap predictor · CIFAR-10 (Case 5) | prediction error: 0.15 vs 0.13 | Hurt |
| give sharpness metrics labeled dev data | 1-sharpness · ImageNet ViT (Case 2) | prediction error: 0.48 -> 0.64 | Mixed |
| add consistency encouragement to UDA | UDA · CIFAR-10 4K labels | test error (%): 4.33 -> 3.95 | Helped |
| add consistency encouragement to SAM | EN-B4 SAM · Food101 | test error (%): 6.00 -> 5.77 | Helped |
| form an ensemble of two models trained with consistency encouragement | ResNet18 · Food101 | inconsistency and test error | Helped |
| encourage consistency for both teacher and student in distillation | distillation · Food101 | test error | Helped |
| apply SAM flatness encouragement | standard model · several cases | generalization gap and sharpness | Helped |
| encourage flatness vs consistency | SAM models · several cases | sharpness vs gap | Mixed |
Closest papers by meaning
- Inconsistency-Aware Minimization: Improving Generalization with Unlabeled Data (ICML 2026 · similarity 0.90)
- Train longer, generalize better: closing the generalization gap in large batch training of neural networks (NeurIPS 2017 · similarity 0.89)
- Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Power (ICLR 2025 · similarity 0.86)
- Understanding Generalization through Decision Pattern Shift (NeurIPS 2026 · similarity 0.86)
- Deep Networks Always Grok and Here is Why (ICML 2024 · similarity 0.86)
- Grokking at the Edge of Numerical Stability (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.