Research map / Architectures and efficiency / Spiking networks
Multiplication-Free Parallelizable Spiking Neurons with Efficient Spatio-Temporal Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multiplication-free, channel-wise parallelizable spiking neuron with sawtooth dilations and power-of-two (bit-shift) weights, plus an autoselect GPU acceleration algorithm, evaluated on SHD, sequential CIFAR and DVS-Lip.
Problem
Existing parallelizable spiking neurons (PSN, sliding PSN, masked PSN) rely on dense floating-point multiplications and need a large neuron order to learn long-term dependencies, which costs compute and memory and is unsuitable for resource-constrained neuromorphic hardware.
Approach
Channel-wise dilated 1-D convolution over time with sawtooth dilation rates (1,2,3 cycling) to enlarge temporal receptive field at small order k, with weights quantized to powers of two so that multiplication becomes bit-shift, trained with a whole-tensor straight-through estimator.
Key results
- Achieves 95.50±0.36% on SHD with two-layer FC + learned delay, order k=2 and sawtooth dilations, above PMSN (95.10) and Stochastic PSN (95.01).
- Reaches 91.17% on sequential CIFAR10 and 66.21% on sequential CIFAR100, outperforming PMSN (90.97 / 66.08) and PSN (88.45 / 62.21).
- Estimated computational energy per CIFAR100 image is 10.66 uJ versus 91.62 uJ for PSN (about 9x lower), but training is still somewhat slower than PSN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace LIF neurons with mul-free channel-wise PSN in the SNN-delay architecture | SHD classification · SHD test | accuracy: 95.07±0.24 -> 95.50±0.36 | Helped |
| use mul-free channel-wise PSN instead of PSN | Sequential CIFAR10 classifier · Sequential CIFAR10 | accuracy: 88.45 -> 91.17 | Helped |
| use mul-free channel-wise PSN instead of PMSN | Sequential CIFAR100 classifier · Sequential CIFAR100 | accuracy: 66.08 -> 66.21 | Helped |
| replace PLIF neurons with mul-free channel-wise PSN and remove spiking neurons from pooling layers | Modified Spiking ResNet-18 on DVS-Lip · DVS-Lip | accuracy: 68.1 -> 70.9 | Helped |
| use sawtooth dilation instead of no dilation (ablation) | Mul-free channel-wise PSN · Pixel CIFAR10 | accuracy: 66.21 -> 68.96; 72.97 -> 76.82 | Helped |
| increase neuron order from k=2 to k=4 (ablation) | Mul-free channel-wise PSN · Sequential CIFAR100 | accuracy: 63.25 -> 65.77 | Helped |
| use order k=4 instead of sliding PSN order k=32 (ablation) | Inference on sequential CIFAR100 · Sequential CIFAR100, T=32 | inference memory (MB): 2635 -> 547 | Helped |
| replace PSN multiplications with bit shifts in the neuron dynamics | Mul-free channel-wise PSN · Single CIFAR100 image | total energy (μJ): 91.62 -> 10.66 | Helped |
| use the original STE gradient rather than the redefined whole-function STE (ablation) | Mul-free channel-wise PSN on DVS-Lip · DVS-Lip | accuracy: 1% | Hurt |
| use autoselect implementation instead of PSN | CIFAR100 training, H20, batch size 128, T=8 · CIFAR100 training | training duration (s/epoch): 27.47 -> 40.99 | Hurt |
Surprising findings
- The original straight-through estimator for power-of-two quantization produced NaN gradients and collapsed training on DVS-Lip, motivating the whole-tensor STE redefinition.
- The proposed neuron trains slower than PSN in wall-clock time at larger T despite fewer inference operations, due to quantization overhead and memory reshape costs.
Limitations
- Training remains slightly slower than PSN due to quantization overhead (convolution applied twice) and Vmap memory operations.
- Energy figures are theoretical estimates on 45nm CMOS rather than measured on hardware.
- Sequential CIFAR and DVS-Lip accuracy comparisons rely partly on numbers copied from prior papers with the same architecture.
Experimental setup
Datasets: Spiking Heidelberg Digits (SHD) Sequential CIFAR10 Sequential CIFAR100 Pixel CIFAR10 DVS-Lip
Benchmarks: SHD Sequential CIFAR10 Sequential CIFAR100 DVS-Lip Pixel CIFAR10
Metrics: test accuracy training time (s/epoch) inference memory (MB) energy (uJ/pJ estimate)
Baselines: PSN sliding PSN masked PSN PMSN GLIF PLIF LIF CLIF Stochastic PSN RPSU SpikGRU2+
Builds on: Parallel Spiking Neuron (PSN) sliding PSN masked PSN SpikingJelly Spiking neural network delay (SNN-delay)
Closest papers by meaning
- Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies (NeurIPS 2023 · similarity 0.91)
- Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity (ICML 2025 · similarity 0.91)
- Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculation (ICML 2025 · similarity 0.89)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.89)
- Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips (ICLR 2024 · similarity 0.89)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.