Research map / Architectures and efficiency / RNNs and normalization
Group Whitening: Balancing Learning Efficiency and Representational Capacity
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace BN with GW in the S1-B2 architecture | ResNet and ResNeXt · ImageNet val | top-1 accuracy: 76.23 -> 77.72; 77.69 -> 78.71; 77.01 -> 78.43; 79.29 -> 80.43 | Helped |
| Replace BN with GN | ResNet and ResNeXt · ImageNet val | top-1 accuracy: 76.23 -> 75.71; 77.69 -> 77.20; 77.01 -> 75.69; 79.29 -> 78.00 | Hurt |
| Use GW rather than GN | Faster R-CNN with ResNet-50 + FPN · COCO val2017, bbox AP | AP: 36.62 -> 38.13; 37.86 -> 39.60 | Helped |
| Use GW rather than GN | Mask R-CNN with ResNeXt-101 + FPN · COCO val2017 | bbox AP: 42.18 -> 44.41 | Helped |
| Use GW rather than GN | Mask R-CNN with ResNeXt-101 + FPN · COCO val2017 | mask AP: 37.54 -> 39.17 | Helped |
| Add GW to the stem and second bottleneck BN (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 76.23 -> 77.72 | Helped |
| Add GW to the stem and third bottleneck BN (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 76.23 -> 77.47 | Helped |
| Add GW to the stem and first and second bottleneck BNs rather than only the first or second (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: 77.62 -> 77.45; 77.72 -> 77.45 | Hurt |
| Add a GW layer after the last average pooling (ablation) | ResNet-50 · ImageNet val | top-1 accuracy: +0.10% | Helped |
| Replace all layer normalizations with GW | Transformer · IWSLT14 De-EN | BLEU: 35.02 ± 0.09 -> 35.27±0.06 | Helped |
| Mix batch normalization with group whitening using the S1-B2 architecture | Faster R-CNN trained from scratch · COCO | AP: 25.10% -> 28.37% | Helped |
| Use a large group number (e.g., g=128) rather than a smaller group number (ablation) | GW and GN networks · ImageNet training | training accuracy | Hurt |
Closest papers by meaning
- An Investigation Into the Stochasticity of Batch Whitening (CVPR 2020 · similarity 0.90)
- Iterative Normalization: Beyond Standardization Towards Efficient Whitening (CVPR 2019 · similarity 0.89)
- Four Things Everyone Should Know to Improve Batch Normalization (ICLR 2020 · similarity 0.89)
- Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning (NeurIPS 2021 · similarity 0.89)
- Dynamic Group Normalization: Spatio-Temporal Adaptation to Evolving Data Statistics (CVPR 2025 · similarity 0.86)
- Population Normalization for Federated Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.