Research map / Architectures and efficiency / RNNs and normalization
Delving Into the Estimation Shift of Batch Normalization in a Network
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the second BN in residual bottlenecks with GN | ResNet and ResNeXt · ImageNet val | top-1 accuracy: 76.29 -> 77.40 | Helped |
| Replace BN with GN throughout the network rather than only one BN per bottleneck (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 76.29 -> 75.73 | Hurt |
| Replace the second rather than the first or third BN in each bottleneck with GN (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 77.08 -> 77.40; 76.76 -> 77.40 | Helped |
| Use IN in selected bottlenecks (D4) rather than in every bottleneck (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 74.99 -> 77.19 | Mixed |
| Use IN rather than GN as the batch-free normalization (ablation) | XBNBlock · ImageNet val under disturbed BN statistics | top-1 accuracy | Helped |
| Add GN between groups of BN layers (ablation) | GNBN network · MNIST with distribution shift | test error | Helped |
| Use XBNBlock_GN instead of frozen BN | ResNet-50 + FPN Faster R-CNN · COCO val2017, 4conv1fc head | AP bbox: 36.85 -> 38.47 | Helped |
| Use XBNBlock_GN instead of frozen BN | ResNet-50 Mask R-CNN · COCO val2017, 4conv1fc head | AP mask: 33.68 -> 34.86 | Helped |
| Use XBNBlock_GN rather than BN | ResNet-50 Faster R-CNN trained from scratch · COCO val2017, batch size 2 per GPU | AP: 25.35 -> 27.45 | Helped |
| Use XBNBlock_GN rather than GN-only | ResNet-50 Faster R-CNN trained from scratch · COCO val2017, batch size 2 per GPU | AP: 28.19 -> 27.45 | Hurt |
| Use XBNBlock_GN rather than BN | ResNet-50 Faster R-CNN trained from scratch · COCO val2017, batch size 4 or 8 per GPU | AP: 29.33 -> 30.51; 29.56 -> 30.58 | Helped |
| Replace GN with group whitening in the XBNBlock (ablation) | ResNet-101 · ImageNet val | top-1 accuracy: 77.65 -> 79.18 | Helped |
Closest papers by meaning
- Understanding Batch Normalization (NeurIPS 2018 · similarity 0.91)
- Reducing Divergence in Batch Normalization for Domain Adaptation (AAAI 2025 · similarity 0.90)
- Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization (ICLR 2020 · similarity 0.90)
- Batch Normalization Is Blind to the First and Second Derivatives of the Loss (AAAI 2024 · similarity 0.88)
- Population Normalization for Federated Learning (CVPR 2025 · similarity 0.87)
- Ghost Noise for Regularizing Deep Neural Networks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.