Research map / Architectures and efficiency / Spiking networks
Understanding the Convergence of Direct Training of SNNs with Surrogate Gradients
Zelin Wei, Tianxing Man, Wanli Shi, Bin Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of convergence for direct training of spiking neural networks using surrogate gradients and local zeroth-order estimators, which keep hard spikes in the forward pass.
Problem
Direct training of spiking neural networks is difficult because spike generation uses a non-differentiable hard threshold, so gradients must be approximated by surrogate or perturbation-based derivatives.
Approach
Provides a convergence analysis for direct SNN training methods that keep hard spikes in the forward pass and use surrogate-gradient or local zeroth-order derivatives in the backward pass, unifying the two under one framework (details beyond the abstract are not available).
Experimental setup
Builds on: surrogate gradient method local zeroth-order estimator
Closest papers by meaning
- Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks (NeurIPS 2021 · similarity 0.92)
- SSF: Accelerating Training of Spiking Neural Networks with Stabilized Spiking Flow (ICCV 2023 · similarity 0.91)
- Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
- Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training (NeurIPS 2025 · similarity 0.90)
- Rethinking the Membrane Dynamics and Optimization Objectives of Spiking Neural Networks (NeurIPS 2024 · similarity 0.90)
- Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
