Research map / Optimization / Optimizers
BSO: Binary Spiking Online Optimization Algorithm
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BSO and T-BSO, online optimizers that train binary spiking neural networks by flipping binary weight signs from gradient-momentum products, eliminating latent weights and reducing training memory to be time-independent.
Problem
Training binary spiking neural networks (BSNNs) with BPTT or latent-weight methods incurs large memory overhead from latent full-precision weights and temporal graph storage, conflicting with BSNN efficiency goals.
Approach
Update binary weights directly via flip signals triggered when the product of gradient momentum and weight exceeds a threshold, removing latent weights; T-BSO scales the threshold by a temporally averaged second-order gradient moment, and the method is trained within an online (forward-in-time) framework such as OTTT or NDOT.
Key results
- T-BSO reaches 94.70% on CIFAR-10 with 6 time steps on VGG-11 at 1.20 MB model size, surpassing OTTT (93.73%).
- On CIFAR-100 (VGG-11, 6 steps), T-BSO achieves 74.27%, above Q-SNN's 73.48% with binary weights.
- Training memory for BPTT rises from 2.86GB to 58.18GB as time steps grow from 1 to 35, while BSO and T-BSO stay constant at 2.9GB and 3.1GB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the temporal-aware threshold using second-order gradient moments instead of BSO's fixed threshold (ablation) | BSO · CIFAR-100 | accuracy: 72.57 -> 74.27; 72.34 -> 73.82; 72.15 -> 73.40 | Helped |
| Add the temporal-aware threshold using second-order gradient moments instead of BSO's fixed threshold (ablation) | BSO · CIFAR-10, 6 time steps | accuracy: 92.98 -> 94.70 | Helped |
| Add the temporal-aware threshold using second-order gradient moments instead of BSO's fixed threshold (ablation) | BSO · DVS-CIFAR10, 10 time steps | accuracy: 80.60 -> 81.00 | Helped |
| Use online BSO instead of BPTT | BSNN training on CIFAR-10 · CIFAR-10, 1 to 35 time steps | training memory: 2.86GB to 58.18GB (20.3$\times$) vs 2.9GB | Helped |
| Use online T-BSO instead of BPTT | BSNN training on CIFAR-10 · CIFAR-10, 1 to 35 time steps | training memory: 2.86GB to 58.18GB (20.3$\times$) vs 3.1GB | Helped |
| Set γ across the tested range instead of the default (ablation) | T-BSO · CIFAR-100, T=2 | accuracy: 72.83\% and 73.41\% | No clear effect |
| Vary the momentum decay rates β₁ and β₂ (ablation) | T-BSO · CIFAR-100, T=2 | accuracy: 73.17; 73.40; 73.34; 73.22; 73.31; 73.29 | No clear effect |
| Use online T-BSO instead of STS-T | VGG-11 on GSC · GSC | accuracy: 95.18 -> 96.12 | Helped |
Surprising findings
- Reported BSO/T-BSO accuracy on ImageNet is below SLTT and OTTT, with T-BSO trailing by several points despite the binary model's compression.
Limitations
- Convergence analysis assumes convex losses and bounded gradients, which do not hold for deep spiking networks.
- On ImageNet, T-BSO lags behind SLTT and OTTT in accuracy.
- Introduces additional hyperparameters (gamma, beta1, beta2) and needs ImageNet-specific fine-tuning settings.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet DVS-CIFAR10 Google Speech Commands
Benchmarks: CIFAR-10 CIFAR-100 ImageNet DVS-CIFAR10 Google Speech Commands
Metrics: top-1 accuracy model size (MB) training memory (GB)
Baselines: OTTT NDOT SLTT CBP Q-SNN BPTT ATIF STS-T
Builds on: OTTT NDOT Adam straight-through estimator
Closest papers by meaning
- Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism (AAAI 2025 · similarity 0.92)
- Enhancing Training of Spiking Neural Network with Stochastic Latency (AAAI 2024 · similarity 0.87)
- Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity (ICML 2025 · similarity 0.87)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.87)
- NDOT: Neuronal Dynamics-based Online Training for Spiking Neural Networks (ICML 2024 · similarity 0.87)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.