Research map / Architectures and efficiency / Spiking networks
S$^2$NN: Sub-bit Spiking Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Sub-bit Spiking Neural Networks (S²NNs) that encode SNN convolution kernels with compact codebooks (<1 bit per weight), adding outlier-aware quantization (OS-Quant) and membrane-potential feature distillation (MPFD).
Problem
Scaled-up spiking neural networks are costly to store and compute on resource-limited edge devices, and binary SNNs still carry substantial storage and compute demands; sub-bit compression has not been explored for SNNs.
Approach
Constrain each layer's binary kernels to a compact codebook of 2^eta codewords (exploiting kernel clustering in well-trained BSNNs), mitigate outlier-induced codeword selection bias with IQR-based outlier detection and spatially-aware scaling, and distill directly on membrane potentials via normalized Gram matrices.
Key results
- On CIFAR-10 with ResNet-19, S²NN at 0.67 bit reaches 96.43% accuracy (vs. 95.54% for Q-SNN baseline) at 1.5x smaller size.
- On ImageNet-1K with SDTv3-19M at 0.67 bit, S²NN reaches 68.02% (+4.6 over BestF 63.46%) with 6.8x fewer OPs.
- Ablation on CIFAR-100 (ResNet-19, 0.67 bit): baseline 75.11%, +OS-Quant 75.59%, +FRFD 76.71%, +MPFD 78.77%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add OS-Quant to the S$^2$NN baseline (ablation) | S$^2$NN · CIFAR-100 | accuracy: 75.11 -> 75.59 (+0.48) | Helped |
| add LGKD (ablation) | S$^2$NN with OS-Quant · CIFAR-100 | accuracy: 75.59 -> 75.92 (+0.33) | Helped |
| add FRFD (ablation) | S$^2$NN with OS-Quant · CIFAR-100 | accuracy: 75.59 -> 76.71 (+1.12) | Helped |
| add MPFD (ablation) | S$^2$NN with OS-Quant · CIFAR-100 | accuracy: 75.11 -> 78.77 (+3.66) | Helped |
| remove the Gram matrix from MPFD (ablation) | MPFD · CIFAR-100 | accuracy: 78.77% -> 78.32% | Hurt |
| use 0.67-bit rather than 1-bit weights | S$^2$NN on CIFAR-10 · CIFAR-10 | accuracy: 95.54 -> 96.43 (+0.9) | Helped |
| use 0.67-bit rather than 1-bit weights | S$^2$NN on CIFAR-100 · CIFAR-100 | accuracy: 78.77 -> 78.77 (+0.0) | No clear effect |
| use 0.44-bit rather than 1-bit weights | S$^2$NN on CIFAR-100 · CIFAR-100 | accuracy: 78.77 -> 77.40 (-1.4) | Hurt |
| use 0.67-bit rather than 1-bit weights | S$^2$NN on ImageNet-1K · ImageNet-1K | accuracy: 63.46 -> 68.02 (+4.6) | Helped |
| use 0.44-bit rather than 1-bit weights | S$^2$NN on DVSCIFAR-10 · DVSCIFAR-10 | accuracy: 81.6 -> 81.3 (-0.3) | Hurt |
| set OS-Quant gamma to 0.5 instead of 1.5 (ablation) | OS-Quant · CIFAR-100 | accuracy: 75.59% -> 74.48% | Hurt |
| set OS-Quant gamma to 3.0 instead of 1.5 (ablation) | OS-Quant · CIFAR-100 | accuracy: 75.59% -> 75.08% | Hurt |
Surprising findings
- Reported CIFAR-10 accuracy at 0.44 bit (95.99%) is still above the binary Q-SNN baseline at 1 bit (95.54%).
Limitations
- The paper states that performance is stronger on visual tasks than on non-visual tasks.
- Reported results are single runs with no error bars or statistical significance tests.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K DVSCIFAR10 COCO 2017 ADE20K
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K DVSCIFAR10 COCO 2017 ADE20K
Metrics: top-1 accuracy model size (Mbit) OPs (G) mAP@50 mIoU pixel accuracy
Baselines: BitSNN Q-SNN Q-Spikformer BESTformer FP SNN IR-Net SNN sub-bit (ICCV2021) Bi-Real BiBert BinaryViT ProxConnect++ A&B
Builds on: Binary neural network sub-bit compression (kernel clustering) Leaky Integrate-and-Fire neuron Tukey's fences / IQR outlier detection Feature-based distillation for SNNs (firing-rate based)
Closest papers by meaning
- QP-SNN: Quantized and Pruned Spiking Neural Networks (ICLR 2025 · similarity 0.91)
- Are Conventional SNNs Really Efficient? A Perspective from Network Quantization (CVPR 2024 · similarity 0.91)
- EnOF-SNN: Training Accurate Spiking Neural Networks via Enhancing the Output Feature (NeurIPS 2024 · similarity 0.90)
- Inference-Scale Complexity in ANN-SNN Conversion for High-Performance and Low-Power Applications (CVPR 2025 · similarity 0.90)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.89)
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.