Research map / Architectures and efficiency / RNNs and normalization
ERA: Enhanced Rational Activations
Martin Trimmel, Mihai Zanfir, Richard Hartley, Cristian Sminchisescu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ReLU with ERA activation | CNN · CIFAR10 | test accuracy (%): 77.3 ± 0.1 -> 84.4 ± 0.2 | Improved |
| replace ReLU with ERA activation | FCN · CIFAR10 | test accuracy (%): 52.6 ± 0.1 -> 58.8 ± 0.2 | Improved |
| use ERA instead of the prior rational activation of [29] | CNN · CIFAR10 | test accuracy (%): 82.1 ± 0.2 -> 84.4 ± 0.2 | Improved |
| use ERA instead of the prior rational activation of [29] | FCN · CIFAR10 | test accuracy (%): 55.0 ± 0.2 -> 58.8 ± 0.2 | Improved |
| initialise ERA as Swish instead of random (ablation) | CNN · CIFAR10 | test accuracy (%): 82.5 ± 0.5 -> 84.4 ± 0.2 | Improved |
| initialise ERA as Swish instead of random (ablation) | FCN · CIFAR10 | test accuracy (%): 57.5 ± 0.4 -> 58.8 ± 0.2 | Improved |
| randomly initialised ERA instead of ReLU (ablation) | CNN · CIFAR10 | test accuracy (%): 77.3 ± 0.1 -> 82.5 ± 0.5 | Improved |
| add layer normalization on activation input (ablation) | ERA CNN · CIFAR10 | test accuracy (%): 81.8 ± 0.3 -> 84.1 ± 0.2 | Improved |
| add layer normalization on activation input (ablation) | ERA FCN · CIFAR10 | test accuracy (%): 57.0 ± 0.4 -> 57.9 ± 0.2 | Improved |
| add partial fraction representation (ablation) | ERA CNN · CIFAR10 | test accuracy (%): 84.1 ± 0.2 -> 84.4 ± 0.2 | Improved |
| add partial fraction representation (ablation) | ERA FCN · CIFAR10 | test accuracy (%): 57.9 ± 0.2 -> 58.8 ± 0.2 | Improved |
| replace ReLU with ERA (deg Q 2) at hidden dim 64 | Transformer head · Human3.6M P2 (weak supervision) | MPJPE-PA (mm): 78.8 ±0.9 -> 73.8 ±2.0 | Improved |
Closest papers by meaning
- Rational neural networks (NeurIPS 2020 · similarity 0.85)
- Efficient Activation Function Optimization through Surrogate Modeling (NeurIPS 2023 · similarity 0.84)
- Padé Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks (ICLR 2020 · similarity 0.84)
- Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild (CVPR 2025 · similarity 0.82)
- Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics (NeurIPS 2025 · similarity 0.82)
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (NeurIPS 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
