Research map / Architectures and efficiency / Spiking networks
Towards efficient deep spiking neural networks construction with spiking activity based pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCA, a structured dynamic sparse training framework that prunes and regrows convolutional channels in deep SNNs based on spiking activity, reducing parameters and SynOps while keeping accuracy on CIFAR10/100, DVS-CIFAR10 and Tiny ImageNet.
Problem
Deep SNNs are heavily over-parameterized with redundant units, and existing SNN compression mostly uses unstructured connection pruning that needs specialized hardware support.
Approach
Channel-level (convolutional kernel) structured pruning during training, where channel importance is the average magnitude of membrane potential (spiking activity), pruned channels are regrown based on BN gamma gradients, and L1 regularization guides redundancy identification; the final network is built by physically removing masked channels.
Key results
- On CIFAR-10 VGG16, SCA keeps 9.31% of parameters with -0.88% accuracy change and 28.39% of parameters with +0.53% gain (91.14% ACC at 55.86% connectivity reported in table).
- On CIFAR-100 VGG16, SCA reaches 64.89% accuracy (+0.64%) at 23.52% connectivity.
- On DVS-CIFAR10, SCA keeps 6.95% of parameters with under 1% accuracy loss (72.80% ACC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCA structured channel pruning instead of the unpruned model | VGG16 · CIFAR10 | accuracy: +0.53% | Helped |
| Use SCA structured channel pruning instead of the unpruned model | VGG16 · CIFAR100 | accuracy: +0.64% | Helped |
| Use SCA structured channel pruning instead of the unpruned model | SCA-based network · DVS-CIFAR10 | accuracy: improves by nearly 1% | Helped |
| Use SCA structured channel pruning instead of the unpruned model | VGG16 · Tiny ImageNet | accuracy: +0.03% | Helped |
| Add channel regrowth to pruning (ablation) | SNNVGG16 · CIFAR10 | test accuracy: about a $2\%$ decrease | Helped |
| Replace pruning with random pruning at initialization (ablation) | SNNVGG16 · CIFAR10 | test accuracy | Hurt |
| Remove regrowth and only prune (ablation) | SNNVGG16 · CIFAR10 | test accuracy | Hurt |
Surprising findings
- Pruning to certain ratios sometimes slightly improves accuracy over the dense baseline (e.g., +0.53% on CIFAR-10, +0.64% on CIFAR-100).
Limitations
- Does not excel in compression versus attention-based, Grad R and dendritic methods on some settings, per authors.
- Hardware speedup and energy measured only through parameter and SynOps counts, not on real neuromorphic hardware.
- Ablation and some results rely on specific VGG/PreResNet variants and time steps that differ from baselines.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 DVS-CIFAR10 Tiny ImageNet
Benchmarks: CIFAR-10 CIFAR-100 DVS-CIFAR10 Tiny ImageNet
Metrics: test accuracy accuracy loss connectivity (remaining parameters %) parameter count synaptic operations
Baselines: Attention-based pruning ADMM-based pruning Grad R Dendritic-based pruning ESL-SNNs SD-SNN PAC-based pruning
Builds on: surrogate gradient Deep Rewiring Grad R ESL-SNNs SpikingJelly
Closest papers by meaning
- Improving the Sparse Structure Learning of Spiking Neural Networks from the View of Compression Efficiency (ICLR 2025 · similarity 0.92)
- Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN (ICLR 2024 · similarity 0.90)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.89)
- QP-SNN: Quantized and Pruned Spiking Neural Networks (ICLR 2025 · similarity 0.88)
- P-SPIKESSM: HARNESSING PROBABILISTIC SPIKING STATE SPACE MODELS FOR LONG-RANGE DEPENDENCY TASKS (ICLR 2025 · similarity 0.88)
- The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.