Research map / Architectures and efficiency / RNNs and normalization
SSN: Learning Sparse Switchable Normalization via SparsestMax
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SN's soft combinations with sparse one-normalizer selection using SparsestMax | ResNet-50 · ImageNet validation, batch size 32 | top-1 accuracy (%): 76.9 -> 77.2 | Helped |
| Replace SN's soft combinations with sparse one-normalizer selection using SparsestMax | ResNet-50 · ImageNet inference, batch size 32 | throughput (images/second): 183.509~$\pm$~0.026 -> 216.254~$\pm$~0.376 | Helped |
| Use SSN instead of SN | DeepLab, ResNet-50 · ADE20K validation | mIoU (%): 39.1 -> 39.3 | Helped |
| Use SSN instead of SN | DeepLab, ResNet-50 · Cityscapes test | mIoU (%): 75.7~\vs~75.8 | No clear effect |
| Use SSN¹, fine-tuned from ImageNet-pretrained SSN, instead of SN | ResNet-50 I3D · Kinetics validation, batch size 8, 32 frames | top-1 accuracy (%): 73.5 -> 73.8 | Helped |
| Add GN to the normalizer set (ablation) | SSN, ResNet-50 · ImageNet validation, batch size 32 | top-1 accuracy (%): 77.3% vs 76.8% | Helped |
| Use the two-stage argmaxed-SN structure instead of learning sparse choices with SSN in one stage (ablation) | ResNet-50 · ImageNet validation | top-1 accuracy (%): 76.8% vs 77.2% | Hurt |
| Vary the schedule for increasing radius r so it reaches the inscribed-circle radius at different epochs (ablation) | SSN, ResNet-50 · ImageNet validation | top-1 accuracy (%): 77.2$\pm$0.04% | No clear effect |
| Use the ImageNet-pretrained SN model as the initialization for SSN² rather than the SSN initialization used for SSN¹ (ablation) | ResNet-50 I3D · Kinetics validation, batch size 8, 32 frames | top-1 accuracy (%): 73.8 vs 73.4 | Mixed |
Closest papers by meaning
- Differentiable Learning-to-Normalize via Switchable Normalization (ICLR 2019 · similarity 0.91)
- A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training (NeurIPS 2024 · similarity 0.87)
- SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training (ICML 2026 · similarity 0.87)
- Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training (NeurIPS 2021 · similarity 0.87)
- Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! (ICLR 2023 · similarity 0.86)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.