Research map / Optimization / Optimizers
Calibrated Stochastic Gradient Descent for Convolutional Neural Networks
Li’an Zhuo, Baochang Zhang, Chen Chen, Qixiang Ye, Jianzhuang Liu, David Doermann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase K from 1 to 8 (sigma=0.0001) (ablation) | CCNN-18 · CIFAR-10 | error rate (%): 9.48 -> 7.42 | Improved |
| increase K from 1 to 4 (sigma=0) (ablation) | CCNN-18 · CIFAR-10 | error rate (%): 9.68 -> 7.82 | Improved |
| decrease sigma from 0.001 to 0.0001 (K=4) (ablation) | CCNN-18 · CIFAR-10 | error rate (%): 7.68 -> 7.61 | Improved |
| decrease sigma from 0.001 to 0.0001 (K=1) (ablation) | CCNN-18 · CIFAR-10 | error rate (%): 9.60 -> 9.48 | Improved |
| add Gaussian noise sigma=0.0001 versus sigma=0 (K=8) (ablation) | CCNN-18 · CIFAR-10 | error rate (%): 7.63 -> 7.42 | Improved |
| gradient calibration layers (K=4) instead of Gaussian noise ResNet-18* | ResNet-18 · CIFAR-10 | error rate (%): 9.72 -> 7.61 | Improved |
| replace conv layers with GClayers (WRN-28 to CCNNs3-28) | WRN · CIFAR-10 | error rate (%): 4.00 -> 3.81 | Improved |
| replace conv layers with GClayers (WRN-28 to CCNNs3-28) | WRN · CIFAR-100 | error rate (%): 19.25 -> 19.11 | Improved |
| replace conv layers with GClayers (WRN-40 to CCNNs2-40) | WRN · CIFAR-10 | error rate (%): 4.53 -> 4.62 | Hurt |
| replace conv layers with GClayers (WRN-40 to CCNNs2-40) | WRN · CIFAR-100 | error rate (%): 21.18 -> 21.67 | Hurt |
| CCNNs-18 versus ResNet-18 | ResNet-18 · ImageNet | Top-1 error: 30.7% -> 29.2% | Improved |
| CCNNs-18 versus ResNet-18 | ResNet-18 · ImageNet | Top-5 error: 10.8% -> 10.3% | Improved |
Closest papers by meaning
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (NeurIPS 2019 · similarity 0.87)
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (ICLR 2017 · similarity 0.87)
- Fast and Accurate Stochastic Gradient Estimation (NeurIPS 2019 · similarity 0.87)
- A Guide Through the Zoo of Biased SGD (NeurIPS 2023 · similarity 0.87)
- Stochastic Approximate Gradient Descent via the Langevin Algorithm (AAAI 2020 · similarity 0.87)
- SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs (NeurIPS 2021 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
