Research map / Architectures and efficiency / Spiking networks

P-SPIKESSM: HARNESSING PROBABILISTIC SPIKING STATE SPACE MODELS FOR LONG-RANGE DEPENDENCY TASKS

ICLR 2025 · Architectures and efficiency › Spiking networks · approach: Bio-inspired spiking neuron models · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P-SpikeSSM, a probabilistic spiking state-space model that samples spikes from SSM-derived firing probabilities (SpikeSampler), uses a surrogate gradient, SpikeMixer and ClampFuse layers, and reports SOTA among SNNs on LRA, psMNIST and SC10.

Problem

Scalable spiking neural networks for long-range dependency sequence tasks are limited by scalar LIF membrane state, sequential spike generation with Heaviside functions, and BPTT training costs.

Approach

Replace LIF neurons with an SSM neuron (n-dimensional hidden state, HiPPO-initialized) whose clamped output is used as the Bernoulli spiking probability; spikes are sampled stochastically, the process is computed as a parallel convolution over spikes, and the backward pass uses the expected value E[S_t]=p_s[t] as a surrogate gradient.

Key results

• On psMNIST, P-SpikeSSM reaches 98.4% accuracy, the best among listed SNNs and above the non-spiking Transformer (97.9%) and LSTM (95.1%).
• On LRA, P-SpikeSSM averages higher than other spiking models on Retrieval (88.53), Text (81.20), and Image (82.40), but is below S4 (Original) on ListOps and Pathfinder.
• Energy analysis on ListOps estimates the SNN is 36x more energy efficient than a non-spiking S4 model under the stated ACC/MAC cost assumptions.

Measured effects reported in the paper

Surprising findings

• Replacing the stochastic SpikeSampler with a deterministic LIF neuron in the same architecture gives lower psMNIST accuracy (97.7% vs 98.4%) and slower training (6.1 vs 1.5 min/epoch).
• Without the surrogate gradient the model fails to exceed random accuracy on ListOps.

Limitations

• Energy analysis covers arithmetic operations only, ignores memory I/O and architectural details, and uses standard convolution instead of FFT.
• Performance on ListOps and Pathfinder trails the original S4 and improved S4 results.
• Some components (gelu in SpikeMixer, normalization in ClampFuse, learnable sigma) involve non-spiking operations that are not fully hardware-friendly.

Experimental setup

Datasets: psMNIST Speech Commands (SC10) Long Range Arena (ListOps, Text, Retrieval, Image, Pathfinder) Speech Commands 35-class

Benchmarks: psMNIST SC10 Long Range Arena Speech Commands 35-class

Metrics: accuracy energy estimate (pJ, ACC vs MAC) training time per epoch active neuron ratio

Baselines: S4 Transformer Linformer Performer Spiking LMUFormer BinaryS4D DSD-SNN HSLMU

Builds on: HiPPO S4 LIF neuron model BinaryS4D Spiking LMUFormer

Closest papers by meaning

• SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.92)
• LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning (NeurIPS 2026 · similarity 0.90)
• TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling (AAAI 2024 · similarity 0.90)
• The Dormant Spiking Neuron: A State-Driven Mechanism for Efficient Spiking Neural Networks (NeurIPS 2026 · similarity 0.89)
• SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking (ICLR 2025 · similarity 0.88)
• Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings (ICLR 2024 · similarity 0.88)