Research map / Architectures and efficiency / RNNs and normalization
ErfAct and Pserf: Non-monotonic Smooth Trainable Activation Functions
Koushik Biswas, Sandeep Kumar, Shilpak Banerjee, Ashish Kumar Pandey
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ReLU with ErfAct | AlexNet · MNIST | accuracy: 99.09 $\pm$ 0.10 -> 99.51 $\pm$ 0.10 | Improved |
| Replace ReLU with Pserf | AlexNet · Fashion MNIST | accuracy: 93.22 $\pm$ 0.21 -> 93.82 $\pm$ 0.19 | Improved |
| Replace ReLU with ErfAct | WideResNet 28-10 · Tiny ImageNet | top-1 accuracy: 61.61 $\pm$ 0.47 -> 64.20 $\pm$ 0.51 | Improved |
| Replace ReLU with Pserf | WideResNet 28-10 · Tiny ImageNet | top-1 accuracy: 61.61 $\pm$ 0.47 -> 64.01 $\pm$ 0.49 | Improved |
| Replace ReLU with ErfAct | ShuffleNet V2 2.0x · CIFAR100 | top-1 accuracy: 67.49 -> 73.17 | Improved |
| Replace ReLU with Pserf | ShuffleNet V2 2.0x · CIFAR100 | top-1 accuracy: 67.49 -> 72.91 | Improved |
| Replace ReLU with ErfAct | ShuffleNet V2 2.0x · CIFAR10 | top-1 accuracy: 91.63 -> 93.74 | Improved |
| Replace ReLU with ErfAct | U-Net · Cityscapes | mIOU: 69.32 $\pm$ 0.30 -> 71.29 $\pm$ 0.31 | Improved |
| Replace ReLU with Pserf | U-Net · Cityscapes | mIOU: 69.32 $\pm$ 0.30 -> 71.21 $\pm$ 0.34 | Improved |
| Replace ReLU with ErfAct | SSD300 · Pascal VOC | mAP: 77.2 $\pm$ 0.14 -> 78.2 $\pm$ 0.12 | Improved |
| Replace ReLU with Pserf | SSD300 · Pascal VOC | mAP: 77.2 $\pm$ 0.14 -> 78.2 $\pm$ 0.14 | Improved |
| Replace ReLU with ErfAct | Transformer · WMT-2014 newstest2014 | BLEU: 26.2 $\pm$ 0.15 -> 26.8 $\pm$ 0.11 | Improved |
Closest papers by meaning
- Trainable Highly-expressive Activation Functions (ECCV 2024 · similarity 0.85)
- Adaptive Parametric Activation (ECCV 2024 · similarity 0.82)
- DiGRAF: Diffeomorphic Graph-Adaptive Activation Function (NeurIPS 2024 · similarity 0.81)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.81)
- Global Convergence in Neural ODEs: Impact of Activation Functions (ICLR 2025 · similarity 0.80)
- Exploring the Benefit of Activation Sparsity in Pre-training (ICML 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
