Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Differential Equation Units: Learning Functional Forms of Activation Functions from Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Maxout activations with DEU activations | MLP · Fashion-MNIST | test accuracy: 89.4 -> 89.8 | Helped |
| replace Maxout activations with DEU activations | CNN · Fashion-MNIST | test accuracy: 91.3 -> 91.5 | Helped |
| replace Maxout activations with DEU activations | MLP · MNIST | test accuracy: 98.5 -> 98.3 | Hurt |
| replace Maxout activations with DEU activations | CNN · MNIST | test accuracy: 99.4 -> 99.2 | Hurt |
| replace Maxout activations with DEU activations | ResNet-18 · CIFAR-10 | test accuracy: 92.5 -> 92.5 | No clear effect |
| replace Maxout activations with DEU activations | Preact-ResNet · CIFAR-10 | test accuracy: 92.4 -> 92.3 | Hurt |
| replace Maxout activations with DEU activations | ResNet-Stunted · CIFAR-10 | test accuracy: 91.1 -> 90.7 | Hurt |
| replace ReLU activations with DEU activations | LSTNet · Traffic 6-hour forecast | RSE: 0.499 -> 0.487 | Helped |
| replace ReLU activations with DEU activations | LSTNet · Traffic 12-hour forecast | CORR: 0.851 -> 0.863 | Helped |
| replace ReLU activations with DEU activations | LSTNet · Solar output 6-hour forecast | RSE: 0.270 -> 0.269 | Hurt |
| replace ReLU activations with DEU activations | LSTNet · Solar output 12-hour forecast | RSE: 0.433 -> 0.327 | Helped |
| replace ReLU activations with DEU activations | LSTNet · Electricity 12-hour forecast | RSE: 0.104 -> 0.100 | Helped |
Closest papers by meaning
- Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations (AAAI 2024 · similarity 0.85)
- Global Convergence in Neural ODEs: Impact of Activation Functions (ICLR 2025 · similarity 0.84)
- Derivative-enhanced Deep Operator Network (NeurIPS 2024 · similarity 0.84)
- Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics (NeurIPS 2025 · similarity 0.83)
- ReLUs Are Sufficient for Learning Implicit Neural Representations (ICML 2024 · similarity 0.83)
- MgNO: Efficient Parameterization of Linear Operators via Multigrid (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.