Research map / Architectures and efficiency / RNNs and normalization
Generalized Batch Normalization: Towards Accelerating Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard deviation and mean with SQD1 (ablation) | BN in ResNet20 · CIFAR-100, no augmentation, LR 0.1, BS 256 | test error rate (%): 58.38 ($\pm$0.28) -> 46.50 ($\pm$0.95) | Helped |
| Replace standard deviation and mean with RSD and its paired statistic (ablation) | BN in ResNet20 · CIFAR-100, data augmentation, LR 0.1, BS 1024 | test error rate (%): 52.64 ($\pm$0.55) -> 44.58 ($\pm$0.30) | Helped |
| Replace standard deviation and mean with RBD (ablation) | BN in ResNet20 · CIFAR-10, no augmentation, LR 0.1, BS 256 | test error rate (%): 22.99 ($\pm$0.64) -> 15.75 ($\pm$0.15) | Helped |
| Replace standard deviation and mean with SQD3 (ablation) | BN in LeNet · CIFAR-10, no augmentation, LR 0.1, BS 256 | test error rate (%): 29.92 ($\pm$0.71) -> 28.37 ($\pm$0.24) | Helped |
| Replace standard deviation and mean with SQD3 (ablation) | BN in LeNet · SVHN, no augmentation, LR 0.1, BS 1024 | test error rate (%): 9.918 ($\pm$0.17) -> 9.447 ($\pm$0.16) | Helped |
| Replace standard deviation and mean with alternative deviation/statistic pairs (ablation) | BN in ResNet20 · CIFAR-10, data augmentation, LR 0.1, BS 256 | test error rate (%): 11.69 ($\pm$0.19) vs 11.76 ($\pm$0.19) | No clear effect |
| Use asymmetric RSD or SQD rather than symmetric SD (ablation) | GBN followed by ReLU · MNIST, LeNet | convergence rate | Helped |
| Use SQD1 rather than standard BN (ablation) | BN in LeNet · MNIST | test error rate | Helped |
| Use WCD centering at the supremum with ReLU (ablation) | GBN · ReLU networks | trainability | Hurt |
Closest papers by meaning
- Understanding Batch Normalization (NeurIPS 2018 · similarity 0.92)
- Double Forward Propagation for Memorized Batch Normalization (AAAI 2018 · similarity 0.91)
- Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization (ICLR 2022 · similarity 0.91)
- Four Things Everyone Should Know to Improve Batch Normalization (ICLR 2020 · similarity 0.91)
- Reducing Divergence in Batch Normalization for Domain Adaptation (AAAI 2025 · similarity 0.89)
- Ghost Noise for Regularizing Deep Neural Networks (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.