Research map / Probabilistic and scientific modeling / Variational and generative sampling
An Adaptive Empirical Bayesian Method for Sparse Deep Learning
Wei Deng, Xiao Zhang, Faming Liang, Guang Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt SSGL priors with stochastic approximation instead of fixing the latent priors (ablation) | SGHMC on MNIST · MNIST | accuracy: 99.47 -> 99.59 | Improved |
| adapt SSGL priors with stochastic approximation instead of fixing the latent priors (ablation) | SGHMC on FMNIST · FMNIST | accuracy: 92.88 -> 93.01 | Improved |
| add stochastic approximation updates for latent variables rather than the fixed-step SGLD-EM update (ablation) | SGLD-EM · Linear regression simulation test set (v0=0.1, σ=1) | MAE: 2.07 -> 1.54 | Improved |
| add stochastic approximation updates for latent variables rather than the fixed-step SGLD-EM update (ablation) | SGLD-EM · Linear regression simulation test set (v0=0.1, σ=1) | MSE: 6.94 -> 4.42 | Improved |
| add adaptive latent-variable updates to SGLD (ablation) | SGLD · Linear regression simulation test set (v0=0.1, σ=1) | MAE: 7.72 -> 1.54 | Improved |
| add adaptive latent-variable updates to SGHMC (ablation) | SGHMC on DA-MNIST · DA-MNIST | accuracy: 99.63 -> 99.75 | Improved |
| add adaptive latent-variable updates to SGHMC (ablation) | SGHMC on DA-FMNIST · DA-FMNIST | accuracy: 94.29 -> 94.38 | Improved |
| add simulated annealing to SGHMC-SA compression (ablation) | SGHMC-SA · CIFAR10 ResNet20 (90% sparsity) | accuracy: 91.45 -> 91.68 | Improved |
| add adaptive stochastic approximation to annealed SGHMC compression (ablation) | A-SGHMC · CIFAR10 ResNet20 (90% sparsity) | accuracy: 90.59 -> 91.68 | Improved |
| add adaptive stochastic approximation to annealed SGHMC compression (ablation) | A-SGHMC · CIFAR10 ResNet20 (30% sparsity) | accuracy: 94.07 -> 94.23 | No clear effect |
| increase target sparsity from 30% to 90% (ablation) | A-SGHMC-SA · CIFAR10 ResNet20 | accuracy: 94.23 -> 91.68 | Hurt |
| use SGHMC-SA instead of Vanilla or Dropout | CNN · MNIST adversarial examples (ζ=0.5) | accuracy: 11% higher than Vanilla and Dropout; 1% higher than SGHMC | Improved |
Closest papers by meaning
- Scalable Bayesian Learning with posteriors (ICLR 2025 · similarity 0.85)
- Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo (ICLR 2025 · similarity 0.85)
- Learning to Explore for Stochastic Gradient MCMC (ICML 2024 · similarity 0.85)
- An Iterative Min-Min Optimization Method for Sparse Bayesian Learning (ICML 2024 · similarity 0.85)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.84)
- Block Sparse Bayesian Learning: A Diversified Scheme (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
