Research map / Architectures and efficiency / RNNs and normalization
SAU: Smooth Activation Function Using Convolution with Approximate Identities
Koushik Biswas, Sandeep Kumar, Shilpak Banerjee, Ashish Kumar Pandey
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ReLU with SAU | ShuffleNet V2 2.0x · CIFAR-100 | top-1 accuracy: 67.11 -> 72.23 | Improved |
| replace ReLU with SAU | ShuffleNet V2 1.0x · CIFAR-100 | top-1 accuracy: 64.57 -> 68.18 | Improved |
| replace ReLU with SAU | ShuffleNet V2 0.5x · CIFAR-100 | top-1 accuracy: 62.02 -> 64.39 | Improved |
| replace ReLU with SAU | EfficientNet B0 · CIFAR-10 | top-1 accuracy: 95.01 -> 96.07 | Improved |
| replace ReLU with SAU | MobileNet V2 · CIFAR-10 | top-1 accuracy: 94.07 -> 95.25 | Improved |
| replace ReLU with SAU | WideResNet 28-10 · Tiny ImageNet | top-1 accuracy: 61.61 -> 63.20 | Improved |
| replace ReLU with SAU | VGG-16 · MNIST | accuracy: 99.05 -> 99.28 | Improved |
| replace ReLU with SAU | VGG-16 · SVHN | accuracy: 95.09 -> 95.41 | Improved |
| replace ReLU with SAU | SSD300 VGG-16 · Pascal VOC | mAP: 77.2 -> 77.7 | Improved |
| replace ReLU with SAU | U-Net · Cityscapes | mIoU: 69.32 -> 71.02 | Improved |
| replace ReLU with SAU | U-Net · Cityscapes | pixel accuracy: 79.60 -> 81.11 | Improved |
| replace ReLU with SAU | Transformer · WMT14 En-De newstest2014 | BLEU: 26.2 -> 26.7 | Improved |
Closest papers by meaning
- Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics (NeurIPS 2025 · similarity 0.84)
- ReLUs Are Sufficient for Learning Implicit Neural Representations (ICML 2024 · similarity 0.84)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.83)
- ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models (ICLR 2024 · similarity 0.83)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.83)
- Activation-Descent Regularization for Input Optimization of ReLU Networks (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
