Research map / Architectures and efficiency / Spiking networks
Rethinking Spiking Neural Networks from an Ensemble Learning Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Treats an SNN as an ensemble of temporal subnetworks and improves it via learnable membrane potential smoothing across timesteps plus KL-based guidance between adjacent timesteps, reporting gains on speech, neuromorphic vision, and point cloud tasks.
Problem
SNNs have limited performance, partly because membrane potential initial states differ greatly across timesteps, making per-timestep subnetwork outputs unstable (especially first timesteps) and suffering temporal gradient vanishing.
Approach
View an SNN as an ensemble of temporal subnetworks; adaptively smooth each timestep's initial membrane potential using the previous smoothed potential with a learnable layer-shared coefficient (also creating temporal residual pathways), and distill from the later adjacent subnetwork to the earlier one with a randomly-dropped guidance loss.
Key results
- Achieves 83.20% accuracy on CIFAR10-DVS with only 4 timesteps (VGGSNN, with data augmentation).
- On DVS-Gesture, reaches 93.23% (VGG-9) and 94.44% (SpikingResformer-Ti) with 5 timesteps.
- Reaches 91.19% on SHD with a lightweight recurrent network, improving DH-LIF baseline from 89.86%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add membrane potential smoothing (ablation) | VGG-9, ResNet-18, and SpikingResformer · CIFAR10-DVS and DVS-Gesture | accuracy: $74.80_{+0.83}$; $68.07_{+1.34}$; $78.45_{+0.85}$; $89.93_{+2.08}$; $82.99_{+2.43}$; $91.32_{+0.69}$ | Helped |
| add temporally adjacent subnetwork guidance (ablation) | VGG-9, ResNet-18, and SpikingResformer · CIFAR10-DVS and DVS-Gesture | accuracy: $76.23_{+2.26}$; $69.33_{+2.60}$; $79.23_{+1.63}$; $91.32_{+3.47}$; $84.84_{+4.28}$; $93.06_{+2.43}$ | Helped |
| add both membrane potential smoothing and temporally adjacent subnetwork guidance (ablation) | VGG-9, ResNet-18, and SpikingResformer · CIFAR10-DVS and DVS-Gesture | accuracy: $\textbf{76.77}_{+2.80}$; $\textbf{70.03}_{+3.30}$; $\textbf{80.60}_{+3.00}$; $\textbf{93.23}_{+5.38}$; $\textbf{85.30}_ | Helped |
| add membrane potential smoothing (ablation) | PLIF, CLIF, RNN-LIF, and DH-LIF · CIFAR10-DVS and SHD | accuracy: $75.10_{+0.27}$; $75.97_{+1.00}$; $83.04_{+1.17}$; $90.33_{+0.47}$ | Helped |
| add membrane potential smoothing only (ablation) | PointNet++ SNN · ModelNet10, inference T=2 | accuracy: $94.23_{+0.48}$ | Helped |
| add temporally adjacent subnetwork guidance only (ablation) | PointNet++ SNN · ModelNet10, inference T=2 | accuracy: $94.05_{+0.30}$ | Helped |
| add both membrane potential smoothing and temporally adjacent subnetwork guidance (ablation) | PointNet++ SNN · ModelNet10, inference T=2 | accuracy: $\textbf{94.54}_{+0.79}$ | Helped |
| use membrane potential smoothing and temporally adjacent subnetwork guidance | SNN · CIFAR10-DVS, inference T=1 | accuracy: 10.00 -> 66.60 | Helped |
| use membrane potential smoothing and temporally adjacent subnetwork guidance | Recurrent SNN · SHD | accuracy: 89.86 -> 91.19 | Helped |
| use membrane potential smoothing and temporally adjacent subnetwork guidance | PointNet++ SNN · ModelNet10 and ModelNet40, T=2 | accuracy: 92.98 -> 94.54; 88.46 -> 91.13 | Helped |
| initialize membrane potential randomly instead of using the vanilla initialization (ablation) | SNN · CIFAR10-DVS, inference T=1 to T=5 | accuracy: 11.90; 61.50; 71.40; 72.30; 74.90 | Mixed |
| set the guidance temperature to 0.5 rather than a value greater than 1 (ablation) | VGG-9 · CIFAR10-DVS | accuracy | Hurt |
Surprising findings
- Vanilla SNN achieves only 10.00% accuracy at inference timestep 1 on CIFAR10-DVS, indicating the first-timestep output is nearly uninformative.
- Reducing cross-timestep differences rather than increasing ensemble diversity improves performance, even though diversity metric drops only 3.05%.
Limitations
- Gradient errors from surrogate gradients cause early-layer smoothing coefficients not to converge to a common value.
- Guidance loss adds training overhead proportional to number of timesteps; the drop-probability and temperature settings are chosen empirically.
Experimental setup
Datasets: CIFAR10-DVS DVS-Gesture N-Caltech101 SHD ModelNet10 ModelNet40 CIFAR10 CIFAR100
Benchmarks: CIFAR10-DVS DVS-Gesture N-Caltech101 SHD ModelNet10 ModelNet40
Metrics: accuracy
Baselines: Ternary Spike SLTT NDOT SSNN SLT CLIF SpikingResformer SDT TRT TIM DH-LIF TC-LIF
Builds on: leaky integrate-and-fire neuron knowledge distillation surrogate gradient PLIF CLIF
Closest papers by meaning
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks (NeurIPS 2024 · similarity 0.92)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.90)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.90)
- Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers (NeurIPS 2025 · similarity 0.90)
- Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings (ICLR 2024 · similarity 0.89)
- Training High Performance Spiking Neural Network by Temporal Model Calibration (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.