Research map / Time series / Foundation models for time series
Advancing Spiking Neural Networks for Sequential Modeling with Central Pattern Generators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CPG-PE, a parameter-free, spike-form positional encoding for SNNs inspired by central pattern generators, showing sinusoidal PE is a special solution of a CPG oscillator, and improves SNNs on time-series forecasting, text classification and image classification.
Problem
Spiking neural networks lack an effective, hardware-friendly, spike-form positional encoding, which hinders their use on sequential tasks such as text classification and time-series forecasting.
Approach
Generate spike-form positional encodings from a group of CPG-inspired neurons firing periodic spike trains (thresholded cosine/sine of t/tau^(i/N)), concatenated with the input spikes and mapped back by a linear layer; the authors also show sinusoidal PE is a particular solution of a 2-neuron coupled CPG membrane dynamics.
Key results
- Spikformer w/ CPG-PE reaches 79.41 average accuracy across 6 text classification benchmarks, vs 77.25 for Spikformer w/o PE and 78.44 for Float-PE, and above fine-tuned BERT's 84.33 avg is not exceeded; it beats all SNN variants.
- On CIFAR10-DVS, CIFAR100 and CIFAR10, Spikformer w/ CPG-PE attains 78.06, 77.27 and 94.82 accuracy, the best among SNN variants.
- Repetition rate of positional spike matrices is 12.19% for RPE versus 0.00% for CPG-PE on CIFAR10-DVS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CPG-PE to Spikformer | Spikformer · Text classification (6-benchmark average) | accuracy: 77.25 -> 79.41 | Helped |
| Replace CPG-PE with a random spike matrix (ablation) | Spikformer · Text classification (6-benchmark average) | accuracy: 79.41 -> 77.23 | Hurt |
| Use CPG-PE instead of no PE | Spikformer · Image classification (3-benchmark average) | accuracy: 81.25 -> 83.38 | Helped |
| Reduce CPG-PE model parameters to be comparable to Spikformer w/o PE (ablation) | Spikformer w/ CPG-PE · Image classification (3-benchmark average) | accuracy: 83.38 -> 83.17 | Hurt |
| Add CPG-PE to Spikformer | Spikformer · Image classification (ImageNet) | accuracy: 69.46 -> 71.17 | Helped |
| Vary the base period τ across 100, 1000, 5000, and 10000 (ablation) | Spikformer w/ CPG-PE · Time-series forecasting and image classification | performance | No clear effect |
| Increase the number of CPG pairs N (ablation) | Spikformer w/ CPG-PE · Time-series forecasting and image classification | performance | Helped |
| Replace CPG-PE with RPE | Spikformer · Image classification (3-benchmark average) | accuracy: 83.38 -> 83.12 | Hurt |
Surprising findings
- Relative PE in Spikformer exhibits a 12.19% repetition rate across positions, undermining uniqueness.
- Image classification gains are modest compared with sequential tasks, attributed to unordered image patches.
Limitations
- Not directly applicable to SNNs without a sequence-length dimension, such as SEW-ResNet.
- Image data requires patching, which may hurt local spatial feature capture.
- Image gains are relatively modest compared with time-series and text gains.
Experimental setup
Datasets: METR-LA PEMS-BAY Solar-energy Electricity MR SST-2 SST-5 Subj ChnSenti Waimai CIFAR10 CIFAR100
Benchmarks: METR-LA PEMS-BAY Solar-energy Electricity MR SST-2 SST-5 Subj ChnSenti Waimai CIFAR10 CIFAR10-DVS
Metrics: R^2 RSE accuracy repetition rate
Baselines: SpikeTCN SpikeRNN Spikformer w/o PE Spikformer w/ RPE Spikformer w/ Float-PE Fine-tuned BERT Vision Transformer
Builds on: Spikformer Spike-driven Transformer Transformer sinusoidal positional encoding surrogate gradient training
Closest papers by meaning
- Toward Relative Positional Encoding in Spiking Transformers (NeurIPS 2025 · similarity 0.91)
- Efficient and Effective Time-Series Forecasting with Spiking Neural Networks (ICML 2024 · similarity 0.87)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.87)
- A Scalable, Causal, and Energy Efficient Framework for Neural Decoding with Spiking Neural Networks (NeurIPS 2025 · similarity 0.87)
- SpikF: Spiking Fourier Network for Efficient Long-term Prediction (ICML 2025 · similarity 0.87)
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.