Research map / Probabilistic and scientific modeling / Bayesian optimization
Bayesian Optimization for Sparse Neural Networks With Trainable Activation Functions
Mohamed Fakhfakh, Lotfi Chaâri
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the trainable MMeLU activation to the Bayesian sparse optimizer (vs fixed ReLU) (ablation) | Bayesian sparse CNN_1 · COVID-CT | accuracy: 0.84 -> 0.90 | Improved |
| add the trainable MMeLU activation to the Bayesian sparse optimizer (vs fixed ReLU) (ablation) | Bayesian sparse CNN_2 · COVID-CT | accuracy: 0.88 -> 0.91 | Improved |
| use trainable MMeLU instead of ReLU+ in Bayesian training (ablation) | Bayesian sparse CNN_1 · Fashion-MNIST | accuracy: 0.92 -> 0.93 | No clear effect |
| use trainable MMeLU instead of ReLU+ in Bayesian training (ablation) | Bayesian sparse CNN_1 · CIFAR-10 | accuracy: 0.90 -> 0.91 | No clear effect |
| use trainable MMeLU instead of ReLU+ (ablation) | Bayesian sparse deep CNN · Fashion-MNIST | accuracy: 0.93 -> 0.94 | Improved |
| Bayesian MMeLU instead of Adam with ReLU | CNN_1 · COVID-CT | accuracy: 0.73 -> 0.90 | Improved |
| Bayesian MMeLU instead of Adam with MeLU | CNN_1 · COVID-CT | accuracy: 0.77 -> 0.90 | Improved |
| Bayesian MMeLU instead of Adam with ReLU | CNN_1 · Fashion-MNIST | accuracy: 0.90 -> 0.93 | Improved |
| Bayesian MMeLU instead of Adam with ReLU | CNN_1 · CIFAR-10 | accuracy: 0.87 -> 0.91 | Improved |
| Bayesian MMeLU instead of Adam with ReLU | deep CNN · Fashion-MNIST | accuracy: 0.89 -> 0.94 | Improved |
| trainable parameters add compute over ReLU+ on small CNN | CNN_1 · COVID-CT | training time (min): 40 -> 46.21 | Hurt |
| Bayesian MMeLU time versus Adam ReLU | CNN_1 · Fashion-MNIST | training time (min): 145.4 -> 111.6 | Improved |
Closest papers by meaning
- Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inference (ICML 2023 · similarity 0.89)
- Efficient Activation Function Optimization through Surrogate Modeling (NeurIPS 2023 · similarity 0.88)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.88)
- Globally Optimal Training of Neural Networks with Threshold Activation Functions (ICLR 2023 · similarity 0.88)
- Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality (NeurIPS 2025 · similarity 0.86)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
