Research map / Architectures and efficiency / RNNs and normalization
Stochastic Adaptive Activation Function
Kyungsu Lee, Jaeseung Yang, Haeyun Lee, Jae Youn Hwang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ReLU with ASH | model 1 · ImageNet 2012 | Top-1 accuracy (%): 76.4 -> 78.5 | Improved |
| Replace ReLU with ASH | model 2 · ImageNet 2012 | Top-1 accuracy (%): 75.6 -> 78.6 | Improved |
| Replace ReLU with ASH | model 3 · ImageNet 2012 | Top-5 accuracy (%): 90.7 -> 94.1 | Improved |
| Replace Swish with ASH | ResNet · CIFAR-10 | accuracy (%): 95.5 -> 95.7 | Improved |
| Replace ReLU with ASH | WRN · CIFAR-10 | accuracy (%): 95.6 -> 96.7 | Improved |
| Replace ReLU with ASH | DenseNet · CIFAR-10 | accuracy (%): 95.7 -> 96.0 | Improved |
| Replace ReLU with ASH | ResNet · CIFAR-100 | accuracy (%): 74.5 -> 76.5 | Improved |
| Replace Swish with ASH | WRN · CIFAR-100 | accuracy (%): 78.9 -> 79.2 | Improved |
| Replace Softplus with ASH | DenseNet · CIFAR-100 | accuracy (%): 84.1 -> 84.6 | Improved |
| Replace Swish with ASH | YOLOv5 · COCO | mAP@50: 74.0 -> 74.8 | Improved |
| Replace Swish with ASH | YOLOv5 · PASCAL VOC detection | mAP@50: 72.6 -> 74.1 | Improved |
| Replace Swish with ASH | U-Net · ADE20K | mIoU (%): 51.3 -> 53.4 | Improved |
Closest papers by meaning
- Learning Specialized Activation Functions With the Piecewise Linear Unit (ICCV 2021 · similarity 0.85)
- Bayesian Optimization for Sparse Neural Networks With Trainable Activation Functions (TPAMI 2024 · similarity 0.85)
- Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics (NeurIPS 2025 · similarity 0.83)
- Exploring the Benefit of Activation Sparsity in Pre-training (ICML 2024 · similarity 0.83)
- Isotropic Activation Functions Enable Deindividuated Neurons and Adaptive Topologies (NeurIPS 2026 · similarity 0.82)
- Rethinking Neural Nonlinearity as Gating (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
