Research map / Architectures and efficiency / RNNs and normalization
Differentiable Dynamic Normalization for Learning Deep Representation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace BN with Dynamic Normalization | ResNet101 · ImageNet val | accuracy (%): 77.8 -> 79.2 | Helped |
| replace BN with Dynamic Normalization | ResNet50 · ImageNet val | accuracy (%): 76.4 -> 78.2 | Helped |
| replace Switchable Normalization with DN | ResNet50 · ImageNet val | accuracy (%): 76.9 -> 78.2 | Helped |
| use IN instead of DN | ResNet101 · ImageNet val | accuracy (%): 79.2 -> 72.2 | Hurt |
| use LN instead of DN | ResNet101 · ImageNet val | accuracy (%): 79.2 -> 75.3 | Hurt |
| use BRN or BKN instead of DN | ResNet50 · ImageNet val | accuracy (%): 78.2 -> 76.3 | Hurt |
| replace BN with DN | ResNet50 · CIFAR10 test | accuracy (%): 95.61 -> 95.81 | Helped |
| use GN instead of DN as depth increases | ResNet50 · CIFAR10 test | accuracy (%): 95.81 -> 90.18 | Hurt |
| replace BN with DN at small per-GPU batch (8,2) | ResNet18 · CIFAR10 test | accuracy (%): 91.55 -> 92.45 | Helped |
| reduce number of GPUs from 8 to 2 (fewer samples aggregated for gradients) | DN ResNet18 · CIFAR10 test | accuracy (%): 93.81 -> 94.95 | Helped |
| reduce per-GPU mini-batch from 8 to 2 | DN ResNet18 · CIFAR10 test | accuracy (%): 93.81 -> 92.45 | Hurt |
| use GN with 32 groups vs other group counts | GN ResNet18 · CIFAR10 test | accuracy (%): 93.67 -> 93.17 | Mixed |
Closest papers by meaning
- Differentiable Learning-to-Normalize via Switchable Normalization (ICLR 2019 · similarity 0.88)
- Exemplar Normalization for Learning Deep Representation (CVPR 2020 · similarity 0.88)
- Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training (NeurIPS 2021 · similarity 0.87)
- Kalman Normalization: Normalizing Internal Representations Across Network Layers (NeurIPS 2018 · similarity 0.87)
- Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes (ICLR 2017 · similarity 0.86)
- Dynamic Group Normalization: Spatio-Temporal Adaptation to Evolving Data Statistics (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.