Research map / Architectures and efficiency / RNNs and normalization
Double Forward Propagation for Memorized Batch Normalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Double-Forward propagation (ablation) | MBN ResNet-56 · CIFAR-10 | test error (%): 6.53 -> 6.23 | Helped |
| add Double-Forward propagation (ablation) | BRN ResNet-56 · CIFAR-10 | test error (%): 6.93 -> 6.46 | Helped |
| add Double-Forward propagation (ablation) | BN ResNet-56 · CIFAR-10 | test error (%): 6.97 -> 6.81 | No clear effect |
| replace BN with MBN (single forward) (ablation) | ResNet-56 · CIFAR-10 | test error (%): 6.97 -> 6.53 | Helped |
| use fixed lambda 0.5 instead of 0 (standard BN) (ablation) | MBN ResNet-56 · CIFAR-10 | test error (%): 6.97 -> 6.45 | Helped |
| use scheduled lambda {0.1,0.5,0.9} instead of fixed lambda (ablation) | MBN ResNet-56 · CIFAR-10 | test error (%): 6.45 -> 6.23 | Helped |
| fixed lambda 0.9 or 0.1 instead of schedule (ablation) | MBN ResNet-56 · CIFAR-10 | test error (%): 6.23 -> 6.81 | Hurt |
| increase memory size from 10 to 40 (ablation) | MBN ResNet-20 · CIFAR-10 | test error (%): 7.87 -> 7.83 | No clear effect |
| replace BN with MBN at batch size 8 | ResNet-56 · CIFAR-10 | test error (%): 19.64 -> 15.37 | Helped |
| replace BN with MBN at batch size 128 | ResNet-56 · CIFAR-10 | test error (%): 6.97 -> 6.23 | Helped |
| replace BN with MBN | ResNet-34 · ImageNet | top-1 error (%): 26.73 -> 25.85 | Helped |
| replace BN with MBN | VGG · CIFAR-100 | top-1 error (%): 27.80 -> 26.79 | Helped |
Closest papers by meaning
- Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization (ICLR 2020 · similarity 0.93)
- Understanding Batch Normalization (NeurIPS 2018 · similarity 0.92)
- Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks (ICML 2016 · similarity 0.91)
- Reducing Divergence in Batch Normalization for Domain Adaptation (AAAI 2025 · similarity 0.89)
- Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion (ICLR 2024 · similarity 0.88)
- Ghost Noise for Regularizing Deep Neural Networks (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.