Research map / Optimization / Implicit bias and generalization
Do We Need Zero Training Loss After Achieving Zero Training Error?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add flooding to MLP w/ batch normalization | MNIST MLP · MNIST | accuracy: 98.60% -> 98.72% | Helped |
| Add flooding to MLP w/ batch normalization | Kuzushiji-MNIST MLP · Kuzushiji-MNIST | accuracy: 92.98% -> 93.80% | Helped |
| Add flooding to ResNet18 w/ weight decay and early stopping | ResNet18 · SVHN | accuracy: 92.99% -> 93.42% | Helped |
| Add flooding to ResNet44 w/ data augmentation and LR decay | ResNet44 · CIFAR-10 | accuracy: 88.05% -> 89.61% | Helped |
| Add flooding to ResNet44 w/ data augmentation and LR decay | ResNet44 · CIFAR-100 | accuracy: 63.38% -> 63.70% | Helped |
| Add flooding | Five-hidden-layer neural network · Two Gaussians, low label noise | accuracy: 90.52% (0.71) -> 92.13% (0.48) | Helped |
| Add flooding | Five-hidden-layer neural network · Spiral, middle label noise | accuracy: 89.94% (0.59) -> 93.90% (0.90) | Helped |
| Add flooding | Five-hidden-layer neural network · Sinusoid, high label noise | accuracy: 78.47% (2.39) -> 85.78% (1.34) | Helped |
| Add flooding to ResNet44 without data augmentation or LR decay | ResNet44 · CIFAR-10 | accuracy: 75.38% -> 75.31% | Hurt |
| Add flooding to ResNet44 without data augmentation or LR decay | ResNet44 · CIFAR-100 | accuracy: 46.00% -> 45.83% | Hurt |
| Add early stopping to training without flooding (ablation) | Synthetic-data neural network · Synthetic datasets | accuracy | Mixed |
| Add flooding to training with early stopping (ablation) | Synthetic-data neural network · Synthetic datasets | accuracy: 90.41% (0.98) -> 92.13% (0.52); 85.85% (2.07) -> 88.15% (0.72); 81.09% (2.23) -> 83.87% (0.65) | Helped |
Closest papers by meaning
- Soft ascent-descent as a stable and flexible alternative to flooding (NeurIPS 2024 · similarity 0.85)
- A Loss Curvature Perspective on Training Instabilities of Deep Learning Models (ICLR 2022 · similarity 0.85)
- Early Stopping in Deep Networks: Double Descent and How to Eliminate it (ICLR 2021 · similarity 0.84)
- Minimizing the Maximal Loss: How and Why (ICML 2016 · similarity 0.84)
- An Improved Analysis of Training Over-parameterized Deep Neural Networks (NeurIPS 2019 · similarity 0.84)
- Grokking at the Edge of Numerical Stability (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.