Research map / Architectures and efficiency / Spiking networks
Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns synaptic delays in deep feedforward SNNs by modeling each synapse as a 1D temporal convolution with a single learnable-position Gaussian kernel (DCLS), reaching state-of-the-art on SHD, SSC and GSC-35 with few parameters.
Problem
Learning synaptic delays jointly with weights in deep spiking neural networks is hard because prior methods either work only for shallow SNNs or estimate delay gradients numerically, and the benefit of delay learning over fixed random delays was not properly tested.
Approach
Model each synaptic connection as a 1D temporal convolution kernel with one non-zero Gaussian element whose position is the delay; learn positions and weights by backpropagation using Dilated Convolution with Learnable Spacings, with the Gaussian std decreased exponentially during training so the kernel becomes one-hot at inference.
Key results
- DCLS-Delays (2L-1KC) reaches 95.07±0.24% on SHD with 0.2M parameters, above the prior best of 94.62% (RadLIF, 3.9M parameters) without recurrence.
- On SSC, DCLS-Delays (3L-2KC) reaches 80.69±0.21% with 2.5M parameters, versus 77.40% for RadLIF.
- Fixed random delays nearly match learned delays in the fully connected setting, but with ~96% (SHD) and ~98% (SSC) sparsity learned delays clearly outperform fixed random delays.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn delays with DCLS instead of using RadLIF without delays | Feedforward SNN · SHD | Top1 accuracy: 94.62\% -> 95.07$\pm$0.24\% | Helped |
| Learn delays with DCLS instead of using RadLIF without delays | Feedforward SNN · SSC | Top1 accuracy: 77.40\% -> 80.69$\pm$0.21\% | Helped |
| Learn delays with DCLS instead of using RadLIF without delays | Feedforward SNN · GSC-35 | Top1 accuracy: 94.51\% -> 95.35$\pm$0.04\% | Helped |
| Replace learned DCLS delays with fixed random delays (ablation) | DCLS-Delays · SHD (fully connected) | Test accuracy | No clear effect |
| Replace learned DCLS delays with fixed random delays (ablation) | DCLS-Delays · SSC (fully connected) | Test accuracy | No clear effect |
| Add learnable delays rather than no delays (ablation) | DCLS-Delays · SHD (fully connected) | Test accuracy | Helped |
| Add learnable delays rather than no delays (ablation) | DCLS-Delays · SSC (fully connected) | Test accuracy | Helped |
| Replace fixed random delays with learned delays (ablation) | DCLS-Delays · SHD (sparse connections) | Test accuracy | Helped |
| Replace fixed random delays with learned delays (ablation) | DCLS-Delays · SSC (sparse connections) | Test accuracy | Helped |
| Decrease σ during training instead of keeping it constant at 0.5 (ablation) | DCLS-Delays · SHD | Test accuracy | Helped |
| Decrease σ during training instead of keeping it constant at 0.5 (ablation) | DCLS-Delays · SSC | Test accuracy | Helped |
| Fix weights and learn only delays instead of learning both (ablation) | DCLS-Delays · SHD (fully connected) | Test accuracy | Hurt |
Surprising findings
- Fixed random uniformly distributed delays perform almost on par with learned delays in fully connected networks, which motivates a sparse-connectivity control test.
- Learning only delays with fixed weights gives acceptable results in the fully connected case but not in the sparse case.
Limitations
- Offline discrete-time training only; cannot handle recurrent connections.
- Maximum delay (kernel size) must be fixed in advance.
- Only evaluated on feedforward fully connected SNNs with vanilla LIF neurons and speech datasets.
Experimental setup
Datasets: Spiking Heidelberg Digits (SHD) Spiking Speech Commands (SSC) Google Speech Commands v0.02 (GSC)
Benchmarks: SHD SSC GSC-35
Metrics: top-1 accuracy number of parameters
Baselines: SNN+Delays Adaptive Delays DL128-SNN-Dloss Dense Conv Delays RadLIF SpikGRU Adaptive SRNN Cuba-LIF EventProp-GeNN STSC-SNN TA-SNN MSAT
Builds on: Dilated Convolution with Learnable Spacings (DCLS) surrogate gradient method SpikingJelly Adam
Closest papers by meaning
- SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models (AAAI 2025 · similarity 0.90)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.90)
- Rethinking Spiking Neural Networks from an Ensemble Learning Perspective (ICLR 2025 · similarity 0.89)
- FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping (NeurIPS 2026 · similarity 0.89)
- Enhancing Training of Spiking Neural Network with Stochastic Latency (AAAI 2024 · similarity 0.89)
- Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.