Research map / Architectures and efficiency / RNNs and normalization
An Investigation Into the Stochasticity of Batch Whitening
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the first BN module with PCA whitening | ResNet-18 ARC_A on ImageNet · ImageNet validation | top-1 accuracy (%): 70.31 -> 59.93 | Hurt |
| estimate population statistics using covariance rather than the whitening matrix (ablation) | ZCA whitening in ResNet-18 ARC_A · ImageNet validation | top-1 accuracy (%): 70.58 -> 70.62 | Helped |
| add a BW layer after average pooling (ablation) | CD-based ResNet-18 ARC_A · ImageNet validation | top-1 accuracy (%): 70.46 -> 70.80 | Helped |
| replace alternating BN modules with CD whitening (ablation) | CD-based ResNet-18 ARC_B · ImageNet validation | top-1 accuracy (%): 70.80 -> 68.15 | Hurt |
| use ItN whitening at ARC_C positions and estimate statistics with covariance (ablation) | ResNet-18 · ImageNet validation | top-1 accuracy (%): 71.33 -> 71.62 | Helped |
| add ItN_Σ at ARC_B positions | ResNet-50 · ImageNet validation, step decay | top-1 accuracy (%): 76.20 -> 77.18 | Helped |
| add ItN_Σ at ARC_C positions | ResNet-50 · ImageNet validation, cosine decay | top-1 accuracy (%): 76.62 -> 77.92 | Helped |
| use full-feature CD whitening with coloring rather than BN with coloring | DCGAN on CIFAR-10 · CIFAR-10 GAN stability | FID: 27.61 -> 24.44 | Helped |
| use coloring recovery rather than scale and shift (ablation) | CD whitening in DCGAN · CIFAR-10 GAN stability | FID: 30.11 -> 24.44 | Helped |
| use group ZCA whitening rather than CD whitening | GAN with scale-and-shift recovery · CIFAR-10 GAN stability | FID: 30.11 -> 26.8 | Helped |
| replace CD whitening with ItN whitening | DCGAN · CIFAR-10 GAN, final evaluation | FID: 22.89 -> 22.5 | Helped |
| estimate population whitening statistics using covariance rather than the whitening matrix (ablation) | batch whitening methods in GAN training · CIFAR-10 GAN | FID | Helped |
Closest papers by meaning
- Whitening and Coloring Batch Transform for GANs (ICLR 2019 · similarity 0.92)
- Stochastic Whitening Batch Normalization (CVPR 2021 · similarity 0.91)
- Group Whitening: Balancing Learning Efficiency and Representational Capacity (CVPR 2021 · similarity 0.90)
- Iterative Normalization: Beyond Standardization Towards Efficient Whitening (CVPR 2019 · similarity 0.90)
- CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization (CVPR 2024 · similarity 0.85)
- Reducing Divergence in Batch Normalization for Domain Adaptation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.