Research map / Architectures and efficiency / Spiking networks
Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Lyapunov Noise Pruning (LNP), a task-agnostic method that prunes a randomly initialized heterogeneous recurrent SNN using Lyapunov exponents and spectral graph sparsification, then trains the sparse model for classification or time-series prediction.
Problem
Sparse recurrent spiking neural networks are usually obtained by task-dependent pruning of trained dense models, which is costly and overfits to the task; the paper seeks a stable sparse heterogeneous RSNN designed without task information.
Approach
Prune a random dense heterogeneous RSNN (HRSNN/CHRSNN with per-neuron membrane time constants) without using task data, using a Lyapunov-exponent-based edge retention probability from a linearized firing-rate model, betweenness-centrality node pruning, edge addition for eigenvector delocalization, and Bayesian optimization of timescales.
Key results
- On Lorenz63, LNP-pruned CHRSNN with BP reaches RMSE 0.682 and VPT 39.15 with about 19.14M average SOPs, versus 1.114 RMSE and 157.33M SOPs for AP-pruned CHRSNN with BP.
- On CIFAR10, LNP ResNet19 (converted) gets accuracy loss -0.04 with 22.18x SOP ratio, while the best prior baseline (IMP) gets 13.29x at -0.04 loss.
- LNP keeps neuron sparsity around 90-94% and synapse sparsity around 94-98% across the CIFAR and prediction settings reported.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · Lorenz63 | RMSE: 1.114 ± 0.051 -> 0.682 ± 0.312 | Helped |
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · Lorenz63 | Avg. SOPs (M): 157.33 ± 18.87 -> 19.14 ± 2.04 | Helped |
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · Google Dataset | RMSE: 1.325±0.566 -> 0.832±0.105 | Helped |
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · Google Dataset | Avg. SOPs (M): 157.33 ± 18.87 -> 19.14 ± 2.04 | Helped |
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · CIFAR10 | SOP Ratio: 45.32 ± 0.58 | Helped |
| Use LNP pruning instead of activity-based pruning | CHRSNN trained with BP · CIFAR100 | SOP Ratio: 50.35 ± 0.64 | Helped |
Surprising findings
- Randomly initialized sparse networks with the same size as LNP output show higher variance and performance that worsens as model size shrinks, suggesting structure matters more for smaller networks.
- The paper reports that accuracy loss for LNP ResNet19 (random) on CIFAR10 is -2.15, indicating a larger drop than other rows, while converted variants are near zero.
Limitations
- Pruning-evaluation pipeline relies on task-specific readout and training after pruning, so task-agnostic claim covers only the pruning step
- Several key reported numbers and tables are internally inconsistent (e.g. identical Google RMSE/VPT values shared across different rows), raising reproducibility concerns
- Lyapunov spectrum and Bayesian optimization steps are described only at a high level; no code or data release is stated
Experimental setup
Datasets: Lorenz63 Rossler Google stock price Wind speed CIFAR10 CIFAR100
Benchmarks: Lorenz63 Rossler Google stock price Wind speed CIFAR10 CIFAR100
Metrics: RMSE valid prediction time (VPT) accuracy accuracy loss SOP ratio neuron sparsity synapse sparsity average SOPs
Baselines: activity-based pruning ADMM LSNN pruning Grad R IMP STDS ESL-SNN random Erdos-Renyi initialization
Builds on: activity-based pruning Lyapunov spectrum of RNN (Engelken) spectral graph sparsification surrogate gradient training
Closest papers by meaning
- Improving the Sparse Structure Learning of Spiking Neural Networks from the View of Compression Efficiency (ICLR 2025 · similarity 0.91)
- Towards efficient deep spiking neural networks construction with spiking activity based pruning (ICML 2024 · similarity 0.90)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.89)
- QP-SNN: Quantized and Pruned Spiking Neural Networks (ICLR 2025 · similarity 0.89)
- Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings (ICLR 2024 · similarity 0.88)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.