Research map / Architectures and efficiency / Spiking networks
Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes weight sign flipping in binary spiking neural networks and proposes AGMM, a sigmoid-based adaptive gradient modulation that reduces flipping and improves accuracy on static and neuromorphic benchmarks.
Problem
Training binary spiking neural networks (BSNNs) is hard because binary weights and non-differentiable spikes cause frequent weight sign flipping, leaving a large accuracy gap to full-precision SNNs.
Approach
Proposes AGMM, a forward module that scales the feature map by sigmoid of a time-wise scaled spatial average (learnable per-timestep factor), which in backpropagation scales gradient mean and variance down, reducing sign flipping probability.
Key results
- AGMM with ResNet-19 reaches 96.13% (2 timesteps) and 96.33% (4 timesteps) on CIFAR-10, above prior best Q-SNN by 0.59%.
- On CIFAR-100 AGMM achieves 80.25% (2 timesteps) and 80.70% (4 timesteps), 1.43% above Q-SNN; on ImageNet it reaches 64.67% vs. 62.06% for the best prior binary result.
- Energy estimate on CIFAR-100: AGMM-BSNN 0.064 mJ vs. vanilla BSNN 0.066 mJ and FP-SNN 0.330 mJ, with 2.46 MB model size vs. 50.46 MB for FP-SNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add AGMM gradient modulation (ablation) | Vanilla BSNN · CIFAR-100 | test accuracy | Helped |
| Add AGMM gradient modulation (ablation) | Vanilla BSNN · CIFAR-100 | SOPs: 0.056 -> 0.045 | Helped |
| Add AGMM gradient modulation (ablation) | Vanilla BSNN · CIFAR-100 | Energy: 0.066 -> 0.064 | Helped |
| Add AGMM gradient modulation (ablation) | Vanilla BSNN · CIFAR-100 | model size: 2.31 -> 2.46 | Hurt |
| Add AGMM gradient modulation (ablation) | Vanilla BSNN · CIFAR-100 | MACs: 0.027 -> 0.028 | Hurt |
| Use AGMM instead of the prior direct-train ResNet-19 method | ResNet-19 BSNN · CIFAR-10 | accuracy: 95.54% -> 96.13% | Helped |
| Use AGMM instead of the prior direct-train ResNet-19 method | ResNet-19 BSNN · CIFAR-100 | accuracy: 78.82% -> 80.25% | Helped |
| Use AGMM instead of the prior direct-train ResNet-18 method | ResNet-18 BSNN · ImageNet | accuracy: 64.67% | Helped |
| Use AGMM instead of the prior VGGSNN method | VGGSNN BSNN · DVS-Gesture | accuracy: 97.92% | No clear effect |
| Use AGMM instead of the prior VGGSNN method | VGGSNN BSNN · DVS-CIFAR10 | accuracy: 81.60% -> 82.40% | Helped |
| Use AGMM with 4 timesteps instead of 2 | ResNet-19 AGMM · CIFAR-10 | accuracy: 96.13% -> 96.33% | Helped |
| Use AGMM with 4 timesteps instead of 2 | ResNet-19 AGMM · CIFAR-100 | accuracy: 80.25% -> 80.70% | Helped |
Surprising findings
- AGMM-BSNN's training accuracy is lower than FP-SNN's, yet its test accuracy is comparable, which the authors read as better generalization.
Limitations
- Theoretical justification assumes gradients are normally distributed and approximates away the second term of the sigmoid derivative.
- Evaluated only on image classification tasks; extension to large-scale or language SNNs is left to future work.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet DVS-Gesture DVS-CIFAR10
Benchmarks: CIFAR-10 CIFAR-100 ImageNet DVS-Gesture DVS-CIFAR10
Metrics: top-1 accuracy weight sign flipping ratio firing rate SOPs energy (mJ) model size (MB)
Baselines: Q-SNN Bit-SNN WS-DR vanilla BSNN FP-SNN
Builds on: Bi-Real Net STBP XNOR-Net Q-SNN
Closest papers by meaning
- BSO: Binary Spiking Online Optimization Algorithm (ICML 2025 · similarity 0.92)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
- Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network (ICML 2024 · similarity 0.88)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.88)
- Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks (AAAI 2025 · similarity 0.88)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.