Research map / Architectures and efficiency / Spiking networks
Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time
Duc Anh Nguyen, Ernesto Araya, Adalbert Fono, Gitta Kutyniok
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of discrete-time leaky integrate-and-fire spiking neural networks: universal approximation with explicit neuron counts, a tight bound on the number of linear/constant regions that grows quadratically in latency T, and experiments on bottleneck width and hyperplane shifts.
Problem
Expressivity of spiking neural networks in discrete time (LIF neurons with direct encoding and membrane potential output) is theoretically poorly understood, especially how latency and depth affect the input-space partitioning and approximation.
Approach
Formalizes discrete-time LIF-SNNs as piecewise constant functions on polyhedral regions, shows that each first-layer neuron induces parallel hyperplanes across time (giving at most (T^2+T+2)/2 regions per neuron), and derives universal approximation with explicit width bounds; depth adds no new activation regions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase SNN latency | Discrete-time LIF-SNN with a 20-neuron first hidden-layer bottleneck · CIFAR10 training set | training accuracy | Improved |
| Widen subsequent hidden layers from 20 to 200 neurons | Low-latency SNN with a 20-neuron first hidden-layer bottleneck · CIFAR10 training set | training accuracy gain: approximately 7% | Improved |
| Widen subsequent hidden layers from 20 to 200 neurons | SNN with T=4 or T=8 and a 20-neuron first hidden-layer bottleneck · CIFAR10 training set | training accuracy gain: almost reaching 20% | Improved |
| Widen subsequent hidden layers from 20 to 200 neurons | ANN with a 20-neuron first hidden-layer bottleneck · CIFAR10 training set | training accuracy gain: approximately 21% | Improved |
| Widen subsequent hidden layers from 20 to 200 neurons | SNN with T=16 and a 20-neuron first hidden-layer bottleneck · CIFAR10 training set | training accuracy gain: approximately 30% | Improved |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=1, n1=4, layer 1 · Linearly separable toy dataset | number of regions: 10 -> 6 | Hurt |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=2, n1=2, layer 1 · Linearly separable toy dataset | number of regions: 16 -> 4 | Hurt |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=2, n1=3, layer 1 · Linearly separable toy dataset | number of regions: 35 -> 10 | Hurt |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=2, n1=4, layer 1 · Linearly separable toy dataset | number of regions: 59 -> 12 | Hurt |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=1, n1=3, layer 2 · Linearly separable toy dataset | number of regions: 4 -> 3 | Hurt |
| Train the SNN on the linearly separable toy dataset | Discrete-time LIF-SNN, T=2, n1=2, layer 2 · Linearly separable toy dataset | number of regions: 14 -> 3 | Hurt |
Surprising findings
- Trained networks often have fewer linear regions than at initialization, e.g., T=2 with n1=4 drops from 59 to 12 regions in layer 1 after training on a toy dataset.
- Increasing latency for static tasks does not consistently improve performance, according to the authors' numerical experiments.
Limitations
- Analysis restricted to static inputs, direct encoding, and membrane potential outputs.
- Lower bound on neuron count proven only in the one-dimensional case; multidimensional extension not formally proven.
- Empirical results on CIFAR10 show latency gains that are not consistent for static tasks.
Experimental setup
Datasets: CIFAR10 toy linearly separable dataset in [-1,1]^2
Benchmarks: CIFAR10
Metrics: training accuracy number of linear regions
Baselines: ReLU ANN Heaviside ANN
Builds on: Heaviside ANN expressivity results linear region counting for ReLU networks spike response model
Closest papers by meaning
- TS-LIF: A Temporal Segment Spiking Neuron Network for Time Series Forecasting (ICLR 2025 · similarity 0.89)
- TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling (AAAI 2024 · similarity 0.89)
- Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity (ICML 2025 · similarity 0.89)
- Are Conventional SNNs Really Efficient? A Perspective from Network Quantization (CVPR 2024 · similarity 0.89)
- CLIF: Complementary Leaky Integrate-and-Fire Neuron for Spiking Neural Networks (ICML 2024 · similarity 0.88)
- P-SPIKESSM: HARNESSING PROBABILISTIC SPIKING STATE SPACE MODELS FOR LONG-RANGE DEPENDENCY TASKS (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
