AI Paper Atlas

Research map / Probabilistic and scientific modeling / Bayesian optimization

Bayesian Optimization for Sparse Neural Networks With Trainable Activation Functions

Mohamed Fakhfakh, Lotfi Chaâri

TPAMI 2024 · Probabilistic and scientific modeling › Bayesian optimization · approach: Multi-fidelity hyperparameter optimization · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add the trainable MMeLU activation to the Bayesian sparse optimizer (vs fixed ReLU) (ablation)Bayesian sparse CNN_1 · COVID-CTaccuracy: 0.84 -> 0.90Improved
add the trainable MMeLU activation to the Bayesian sparse optimizer (vs fixed ReLU) (ablation)Bayesian sparse CNN_2 · COVID-CTaccuracy: 0.88 -> 0.91Improved
use trainable MMeLU instead of ReLU+ in Bayesian training (ablation)Bayesian sparse CNN_1 · Fashion-MNISTaccuracy: 0.92 -> 0.93No clear effect
use trainable MMeLU instead of ReLU+ in Bayesian training (ablation)Bayesian sparse CNN_1 · CIFAR-10accuracy: 0.90 -> 0.91No clear effect
use trainable MMeLU instead of ReLU+ (ablation)Bayesian sparse deep CNN · Fashion-MNISTaccuracy: 0.93 -> 0.94Improved
Bayesian MMeLU instead of Adam with ReLUCNN_1 · COVID-CTaccuracy: 0.73 -> 0.90Improved
Bayesian MMeLU instead of Adam with MeLUCNN_1 · COVID-CTaccuracy: 0.77 -> 0.90Improved
Bayesian MMeLU instead of Adam with ReLUCNN_1 · Fashion-MNISTaccuracy: 0.90 -> 0.93Improved
Bayesian MMeLU instead of Adam with ReLUCNN_1 · CIFAR-10accuracy: 0.87 -> 0.91Improved
Bayesian MMeLU instead of Adam with ReLUdeep CNN · Fashion-MNISTaccuracy: 0.89 -> 0.94Improved
trainable parameters add compute over ReLU+ on small CNNCNN_1 · COVID-CTtraining time (min): 40 -> 46.21Hurt
Bayesian MMeLU time versus Adam ReLUCNN_1 · Fashion-MNISTtraining time (min): 145.4 -> 111.6Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy