Research map / Architectures and efficiency / Spiking networks

Efficient Parallel Training Methods for Spiking Neural Networks with Constant Time Complexity

ICML 2025 · Architectures and efficiency › Spiking networks · approach: Sparse efficient spiking sequence models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Fixed-point Parallel Training (FPT), which reformulates LIF neuron dynamics over all T timesteps as a K-iteration fixed-point map so SNN training runs in O(K) sequential time (K=3) while keeping the reset mechanism.

Problem

SNN training with BPTT has O(T) sequential time complexity over timesteps, making training slow and poorly parallelized on GPUs, especially for long sequences.

Approach

Unroll LIF membrane dynamics across all T timesteps into a vector fixed-point equation, replace the Heaviside with a sigmoid surrogate in the forward pass (alpha_f) and a smaller steepness in the backward pass (alpha_b), and iterate K≈3 times with automatic differentiation instead of implicit differentiation. Existing parallel neurons (PSN, PSU/IPSU) are shown to be special cases with fewer iterat

Key results

• FPT with K=3 reaches 85.50±0.22% on DVS-CIFAR10 (VGG-11, T=10), above the listed baselines by over 3 points, and 98.38±0.17% on DVS-Gesture.
• On Amazon Photos with DRSGNN, FPT is over 10x faster than BPTT for T≥32 with no statistically significant accuracy difference (p>0.05).
• FPT with learnable decay matrix reaches 89.53±0.13% on Sequential CIFAR10 and 64.50±0.34% on Sequential CIFAR100, outperforming PSN and IPSU; reported speedup exceeds 100x on GPU for T≥512 in the LIF simulation.

Measured effects reported in the paper

Surprising findings

• Fixed-parameter PSN (no reset) is about 2% less accurate than LIF on sequential CIFAR, while FPT without learnable parameters matches LIF accuracy.
• Increasing K beyond 3 (K=4,5) leaves DVS-Gesture accuracy unchanged at 98.61%.

Limitations

• Memory usage grows with T because all timesteps are processed in parallel, generally exceeding sequential BPTT memory.
• Not applicable to neurons without time-dependent decay (e.g., Integrate-and-Fire).
• Backward pass is not the exact BPTT gradient; the approximation with surrogate steepness alpha_b is heuristic, and small errors accumulate over long T (cosine similarity drops to ~99.5% at T=512).

Experimental setup

Datasets: DVS-CIFAR10 DVS-Gesture ImageNet-100 Amazon Photos Sequential CIFAR10 Sequential CIFAR100 MNIST

Benchmarks: DVS-CIFAR10 DVS-Gesture ImageNet-100 Amazon Photos Sequential CIFAR10 Sequential CIFAR100 MNIST

Metrics: accuracy training time per batch speedup firing rate cosine similarity absolute membrane potential error p-value

Baselines: OTTT SLTT SEENN SSNN SLT SSF T-RevSNN LocalZO NDOT EfficientLIF-Net PSN IPSU

Builds on: BPTT surrogate gradient Parallel Spiking Neuron (PSN) Parallel Spiking Unit (PSU) deep equilibrium models

Closest papers by meaning

• Parallel Training Time-to-First-Spike Spiking Neural Networks (AAAI 2026 · similarity 0.91)
• Multiplication-Free Parallelizable Spiking Neurons with Efficient Spatio-Temporal Dynamics (NeurIPS 2025 · similarity 0.91)
• Faster and Stronger: When ANN-SNN Conversion Meets Parallel Spiking Calculation (ICML 2025 · similarity 0.90)
• Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.90)
• Enhancing Training of Spiking Neural Network with Stochastic Latency (AAAI 2024 · similarity 0.89)
• Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.89)