Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models stochastic spiking neural networks as Event SDEs driven by càdlàg rough paths, derives pathwise gradients of solutions and event times, and trains SSNNs with Marcus signature kernel MMD losses.
Problem
Gradient-based training of stochastic spiking neural networks, where noise affects both spike timing and network dynamics, is hard because implicitly defined spike events make derivatives of trajectories and event times ill-defined under stochastic dynamics.
Approach
Formalise SSNNs as Event SDEs driven by càdlàg rough paths, give sufficient conditions for pathwise differentiability of solutions and event times with a recursive gradient formula, and define robust Marcus signature kernels for characteristic MMD losses on discontinuous paths.
Key results
- Single SLIF neuron input current estimation: backpropagation through the event solver recovers the true c=1.5 after around 600 SGD steps for both noise levels.
- Feed-forward SSNN (4-16-2) weight estimation: test loss reaches practically 0 for sample sizes 512 and 1024.
- Gradients are exact for the numerical solver output (discretise-then-optimise), with no surrogate gradients needed.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the SLIF model using EventSDESolve and signature-kernel MMD | single-neuron stochastic leaky integrate-and-fire model · Input current estimation | MAE (first three average spike times) | Helped |
| train the SLIF model using EventSDESolve and signature-kernel MMD | single-neuron stochastic leaky integrate-and-fire model · Input current estimation | input current estimate | Helped |
| increase the training sample size from 256 to 512 | feed-forward SSNN weight estimation · Synaptic weight estimation | test loss | Helped |
| increase the training sample size from 256 to 1024 | feed-forward SSNN weight estimation · Synaptic weight estimation | test loss | Helped |
| increase the training sample size from 512 to 1024 | feed-forward SSNN weight estimation · Synaptic weight estimation | test loss | No clear effect |
Surprising findings
- For the largest sample size the model reaches the same test loss as with 512 samples, yet its estimated weight matrices differ significantly.
- Setting the gradient with respect to time of the driving Brownian motion to 0 was needed for stability in practice.
Limitations
- Experiments only use discretise-then-optimise; adjoint and reversible solvers not compared.
- Experiments only on spike trains of bounded variation and small synthetic settings.
- Exact gradient formula relies on assumptions (e.g. diffusion orthogonal to event function gradient, nonzero drift in event direction) and time-derivative cancellation can be numerically unstable.
Experimental setup
Datasets: synthetic spike trains simulated from SLIF networks
Metrics: mean absolute error of first three average spike times signature kernel MMD test loss estimated input current c estimated weight matrix vs true
Baselines: surrogate gradient BPTT adjoint method for SNNs
Builds on: Marcus SDEs rough path theory signature kernels diffrax e-prop
Closest papers by meaning
- Sign Gradient Descent-based Neuronal Dynamics: ANN-to-SNN Conversion Beyond ReLU Network (ICML 2024 · similarity 0.88)
- Learning Delays in Spiking Neural Networks using Dilated Convolutions with Learnable Spacings (ICLR 2024 · similarity 0.87)
- Enhancing Training of Spiking Neural Network with Stochastic Latency (AAAI 2024 · similarity 0.87)
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.87)
- Enhancing the Robustness of Spiking Neural Networks with Stochastic Gating Mechanisms (AAAI 2024 · similarity 0.87)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.