Research map / Optimization / Implicit bias and generalization
A Recipe for Global Convergence Guarantee in Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · MNIST, no augmentation | test error (%): 0.41 (0.02) -> 0.38 (0.04) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · MNIST, augmentation | test error (%): 0.34 (0.03) -> 0.28 (0.04) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · CIFAR-10, no augmentation | test error (%): 13.99 (0.17) -> 13.57 (0.32) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · CIFAR-10, augmentation | test error (%): 7.01 (0.18) -> 6.84 (0.16) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · CIFAR-100, no augmentation | test error (%): 41.43 (0.43) -> 40.78 (0.22) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · CIFAR-100, augmentation | test error (%): 27.92 (0.36) -> 27.41 (0.58) | Helped |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · SVHN, no augmentation | test error (%): 4.51 (0.04) -> 4.50 (0.09) | No clear effect |
| Replace base SGD with its two-phase version with guarantee | ResNet-18 · SVHN, augmentation | test error (%): 4.32 (0.06) -> 4.16 (0.16) | Helped |
| Vary the transition fraction τ₀ (ablation) | Two-phase SGD with δ₀ fixed at 0.0001 · Kuzushiji-MNIST | test error (%): 2.10 (0.14); 2.06 (0.07); 2.02 (0.09); 2.01 (0.05) | Mixed |
| Vary the transition fraction τ₀ (ablation) | Two-phase SGD with δ₀ fixed at 0.001 · Kuzushiji-MNIST | test error (%): 2.06 (0.07); 2.11 (0.12); 1.92 (0.06); 2.01 (0.12) | Mixed |
| Vary the transition fraction τ₀ (ablation) | Two-phase SGD with δ₀ fixed at 0.01 · Kuzushiji-MNIST | test error (%): 2.25 (0.11); 2.25 (0.17); 2.25 (0.11); 2.09 (0.08) | Mixed |
| Vary the perturbation scale δ₀ (ablation) | Two-phase SGD with τ₀ fixed at 0.4 · Kuzushiji-MNIST | test error (%): 2.10 (0.14); 2.06 (0.07); 2.25 (0.11) | Mixed |
Closest papers by meaning
- When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work (NeurIPS 2021 · similarity 0.88)
- On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths (ICML 2021 · similarity 0.88)
- Block Coordinate Descent for Neural Networks Provably Finds Global Minima (NeurIPS 2025 · similarity 0.86)
- Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods (ICLR 2025 · similarity 0.86)
- Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel (NeurIPS 2025 · similarity 0.85)
- Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.