Research map / Continual learning / Online and replay-based
Training Binary Neural Networks using the Bayesian Learning Rule
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train binary weights with BayesBiNN instead of STE Adam | Binary neural network · MNIST | test accuracy: 98.85 ± 0.09 -> 98.86 ± 0.05 | No clear effect |
| Train binary weights with BayesBiNN instead of STE Adam | Binary neural network · CIFAR-10 | test accuracy: 93.55 ± 0.15 -> 93.72 ±0.16 | Helped |
| Train binary weights with BayesBiNN instead of STE Adam | Binary neural network · CIFAR-100 | test accuracy: 72.89 ± 0.21 -> 73.68 ±0.31 | Helped |
| Use predictive mean over posterior samples instead of the posterior mode (ablation) | BayesBiNN · MNIST | test accuracy: 98.86 ± 0.05 -> 98.86 ± 0.05 | No clear effect |
| Use predictive mean over posterior samples instead of the posterior mode (ablation) | BayesBiNN · CIFAR-10 | test accuracy: 93.72 ±0.16 -> 93.72 ±0.15 | No clear effect |
| Use predictive mean over posterior samples instead of the posterior mode (ablation) | BayesBiNN · CIFAR-100 | test accuracy: 73.68 ±0.31 -> 73.65 ±0.41 | No clear effect |
| Use the learned posterior from the previous task as the next task's prior (ablation) | BayesBiNN · Permuted MNIST continual learning | test accuracy on past tasks | Helped |
| Use a fixed zero prior instead of the previous task's posterior (ablation) | BayesBiNN · Permuted MNIST continual learning | test accuracy on past tasks | Hurt |
| Increase temperature from $10^{-3}$ through $10^{-7}$ (ablation) | BayesBiNN on MNIST · MNIST | test accuracy: 90.40 ± 0.97 -> 98.41 ± 0.10 | Helped |
| Vary temperature among $10^{-8}$, $10^{-9}$, $10^{-10}$, $10^{-11}$, and $10^{-12}$ (ablation) | BayesBiNN on MNIST · MNIST | test accuracy: 98.82 ± 0.05; 98.81 ± 0.08; 98.86 ± 0.05; 98.86 ± 0.06; 98.84 ± 0.04 | No clear effect |
Closest papers by meaning
- An Empirical study of Binary Neural Networks' Optimisation (ICLR 2019 · similarity 0.89)
- Training Binary Neural Networks via Gaussian Variational Inference and Low-Rank Semidefinite Programming (NeurIPS 2024 · similarity 0.88)
- Critical initialisation in continuous approximations of binary neural networks (ICLR 2020 · similarity 0.87)
- AdaSTE: An Adaptive Straight-Through Estimator To Train Binary Neural Networks (CVPR 2022 · similarity 0.87)
- Kalman Bayesian Neural Networks for Closed-Form Online Learning (AAAI 2023 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.