Research map / Architectures and efficiency / Spiking networks
Improving the Sparse Structure Learning of Spiking Neural Networks from the View of Compression Efficiency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage dynamic sparse training-from-scratch method for spiking neural networks that uses the PQ compressibility index to adaptively set the pruning/regrowth ratio each iteration.
Problem
Deep SNNs are over-parameterized, and existing fully sparse SNN training uses static pruning ratios that cause under- or over-pruning.
Approach
Stage I computes a PQ-index-based compressibility estimate of the current sparse SNN to set an adaptive rewiring ratio; Stage II prunes and regrows connections (momentum-based regrowth) at that ratio, iterating the two stages throughout training starting from an Erdős–Rényi sparse topology.
Key results
- Layer-wise rewiring on CIFAR10 with ResNet19 reaches 92.38% accuracy at 29.72% connection density, above the ~92.2% dense baseline.
- On CIFAR100 with ResNet19 (layer-wise), reaches 70.3% accuracy at 29.48% connection, +1.07% over the non-sparse model.
- On DVS-CIFAR10 with VGGSNN (layer-wise), reaches 78.4% accuracy at 30% connection density (+0.08% vs. dense reference).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train from scratch with sparse connections instead of gradually sparsifying a fully connected network (ablation) | Two-stage sparse structure learning · CIFAR10 and CIFAR100 | Accuracy | Helped |
| Train from scratch with sparse connections instead of gradually sparsifying a fully connected network (ablation) | Two-stage sparse structure learning · CIFAR10 and CIFAR100 | Connection sparsity | Helped |
| Use neuron-wise rewiring instead of layer-wise rewiring (ablation) | Two-stage sparse structure learning · CIFAR10 | Accuracy: 92.48 vs 92.38; 92.1 vs 91.99 | Mixed |
Surprising findings
- Sparse models at roughly 30% connection density slightly exceed the dense SNN accuracy on CIFAR10 and CIFAR100.
Limitations
- Accuracy oscillates and collapses at very high sparsity, and the neuron-wise scheme shows no similar behavior on CIFAR100.
- Evaluated only on small vision benchmarks (CIFAR10, CIFAR100, DVS-CIFAR10).
- Hardware efficiency is argued rather than measured.
Experimental setup
Datasets: CIFAR10 CIFAR100 DVS-CIFAR10
Benchmarks: CIFAR10 CIFAR100 DVS-CIFAR10
Metrics: top-1 accuracy connection density parameter count SOPs
Baselines: ADMM Grad R ESLSNN STDS UPR
Builds on: Grad R sparse evolutionary training PQ index TET loss
Closest papers by meaning
- Towards efficient deep spiking neural networks construction with spiking activity based pruning (ICML 2024 · similarity 0.92)
- Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN (ICLR 2024 · similarity 0.91)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.91)
- QP-SNN: Quantized and Pruned Spiking Neural Networks (ICLR 2025 · similarity 0.90)
- S$^2$NN: Sub-bit Spiking Neural Networks (NeurIPS 2025 · similarity 0.89)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.