Research map / Architectures and efficiency / RNNs and normalization
Multicoated Supermasks Enhance Hidden Networks
Yasuyuki Okoshi, Ángel López Garcı́a-Arias, Kazutoshi Hirose, Kota Ando, Kazushi Kawamura, Thiem Van Chu, Masato Motomura, Jaehoon Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase number of coats from 1 (Hidden Networks) to 7 with Linear density setting (ablation) | ResNet-50 supermask · ImageNet | Top-1 Test Acc. (%): overperforms Hidden Networks by 5.65% | Improved |
| Use multiple coats (N>1) instead of a single supermask (ablation) | Hidden Networks on CIFAR-10 ResNet-18 · CIFAR-10 | Top-1 Test Acc. (%) | Improved |
| Linear coat density scheme instead of Uniform (ablation) | Multicoated Supermask on CIFAR-10 ResNet-18 · CIFAR-10 | Top-1 Test Acc. (%) | Improved |
| Shared scores across coats instead of independent scores per mask (ablation) | Multicoated Supermask on ResNet-50 · ImageNet | Top-1 Test Acc. (%) | Improved |
| Remove weight-initialization advantage: standard Kaiming normal instead of Signed Kaiming Constant as number of coats grows (ablation) | Multicoated Supermask on ImageNet ResNet-50 · ImageNet | Top-1 Test Acc. (%) | No clear effect |
| Increase total density k1 from ~30% to higher values with multicoating on CIFAR-10 (ablation) | Multicoated Supermask ResNet-18 · CIFAR-10 | Top-1 Test Acc. (%) | Mixed |
| Multicoated Supermask (N=7) vs Hidden Networks at similar model size on ImageNet ResNet-101 | ResNet-101 multicoated · ImageNet | Top-1 Test Acc. (%): 76.46% vs 73.85% for Wide ResNet-50 at similar size | Improved |
| Multicoating at 90% sparsity on ResNet-50 (N=7) vs pruning/RigL/MEST baselines | ResNet-50 multicoated supermask · ImageNet | Top-1 Test Acc. (%): 72.5 vs 75.2-76.1 for baselines at same sparsity | Hurt |
Closest papers by meaning
- Signing the Supermask: Keep, Hide, Invert (ICLR 2022 · similarity 0.85)
- Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently (ICLR 2022 · similarity 0.85)
- Supermasks in Superposition (NeurIPS 2020 · similarity 0.84)
- Learning Strict Identity Mappings in Deep Residual Networks (CVPR 2018 · similarity 0.83)
- AlphaNet: Improved Training of Supernets with Alpha-Divergence (ICML 2021 · similarity 0.83)
- Efficient Few-Shot Neural Architecture Search by Counting the Number of Nonlinear Functions (AAAI 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
